Introduction to High-Performance Computing (BSc CS)

Advance Information

This is an overview of the contents of the lecture "Introduction to High-Performance Computing" that will be offered for the first time in Summer Term 2023.

Course Information

Title: High-Performance Computing
Lecturer: Prof. Dr. Christian Plessl
Lecture Number: L.079.05505
Teaching Language: German (but all materials provided in English)

Format

2h lecture
1h theoretical exercise
2h practical exercises/lab per week

Times

Lecture: TBD
Exercise, lab: TBD

Content Overview

This course aims at providing an Introduction to High-Performance Computing. We will start out with an overview of the many layers of parallelism that are available in modern computer architecture, such as, instruction-level parallelism, data-level parallelism (SIMD), multi-cores, multi-processing). We discuss the properties of these layers and learn how we can effectively use the available parallelism in software.

A key objective of this course is to connect parallel computer architecture and performance-oriented software development both in theory and practice. Hence, the course involves a signficiant amount of programming exercises and practial labs where the code is executed on the HPC systems Noctua 1/2 at Paderborn Center for Parallel Computing.

As programming language, we will use Julia, a modern, high-performance, dynamically typed language tailored for scientific computing. Julia has a very rich ecosystem of extension packages for numerical algorithms, data analysis, machine learning, visualization and highly specialized packages for numerous application domains. On the other side, Julia is well suited as an interactive programming environment, which is ideal for learning and explorative research.

For the practical application of the discussed programming paradigms and optimization techniques we will work with case studies from the domains of:

Computational physics
Combinatorial optimization
Image classification
Linear algebra
etc.

Tentative List of Topics

This course is offered for the first time. Hence, the selection of topics is still in flux. The current tentative list of topics is:

A tour of parallel and high-performance computer architecture
Overview of HPC with Julia
Performance Evaluation
Performance Optimization of Serial Programs
Patterns and Models of Parallel Programming
SIMD/Vectorization
Multi-Threaded Programming for Concurrency and Performance
Distributed Memory Programming
Distributed Memory Programming with MPI
Deep Learning
GPU Programming

Materials

Lecture slides
Companion website