Course Title

Introduction to High Performance Computing

Course Code

SDS 406

Course Type

Mandatory

Level

Master’s

Year / Semester

1^st Semester

Instructor’s Name

Assoc. Prof. Giannis Koutsou

ECTS

10

Lectures / week

1

Laboratories / week

1

Course Purpose and Objectives

• Provide the students with the necessary computer programming and software engineering background to solve complex problems using numerical methods.
• Introduce the basic concepts in high performance computing (HPC), understand modern computer architectures, optimization strategies, and parallel programming.

Learning Outcomes

Students will develop the skills necessary to write their own parallel codes and will be trained in common practices used to implement numerical methods that exploit parallelism. Students will develop an understanding of the landscape of HPC architectures.

Prerequisites

None

Course Content

• Week 1: Introduction to HPC – Introduction of concepts such as scaling and parallel efficiency, floating point performance, memory I/O performance.
• Week 2: Cluster computing – Navigating a computer cluster, submitting jobs, distributing parallel workloads.
• Week 3: Shared memory model and parallel processing using OpenMP – Introduction to the fork-join parallel processing model. Shared and private variables. Common pragmas used in OpenMP.
• Week 4: Practical applications using OpenMP – Parallelization of selected linear algebra kernels using OpenMP.
• Week 5: Performance evaluation and performance modelling – Identifying bottlenecks in computational kernels. Kernel intensity and machine intensity.
• Week 6: Optimization – Brief overview of optimization and examples, including cache-locality and vectorization
• Week 7: Distributed memory model and parallel processing using MPI – Introduction to parallel processing with MPI. Collectives and point-to-point operations.
• Week 8: Practical applications using MPI – Distributed memory parallelization of partial differential equations.
• Week 9: MPI custom types and parallel I/O – Binary data formats, derived MPI types, and parallel I/O
• Week 10: Introduction to GPU programming – GPUs and their characteristics. Aspects of GPU programming; threads, blocks, memory hierarchy, memory access patterns
• Weeks 11 & 12: Practical GPU programming – Practical examples implementing GPU kernels
• Week 13: Advanced topics in HPC – An audit of advanced HPC topics, including hybrid MPI/OpenMP programming, multi-GPU programming, advance performance analysis topics
• Week 14: Revision – Revision of taught content, review of solutions based also on student feedback, and preparation for final examination.

Teaching Methodology

• Lectures interleaved with hands-on practical exercises.
• Laboratory work will include completing or fixing codes provided by the instructor, running the codes on an HPC cluster, and visualizing or summarizing the results

Bibliography

• Tutor’s lecture notes and slides
• Optional bibliography: “The Art of HPC, volumes 1 & 2”, V. Eijkhout, ISBN: 9781387387892

Assessment

• Weekly exercises and lab participation
• Three interim homework assignments
• Final homework assignment

Language

English