Advance Information

This is an overview of the contents of the lecture "Introduction to High-Performance Computing" that will be offered for the second time in Summer Term 2024.

Course Information

• Title: High-Performance Computing / Einführung in Hochleistungsrechnen
• Lecturer: Prof. Dr. Christian Plessl
• Lecture Number: L.079.05602
• Teaching Language: German (but all materials provided in English)

Format

• 2h lecture
• 1h theoretical exercise
• 2h practical exercises/lab per week

Times

• Lecture: Tuesdays, 8:55 - 10:25 hrs
• Exercise, lab: Thursdays, 16:15 - 18:45 hrs

Content Overview

This course aims at providing an Introduction to High-Performance Computing. We will start out with an overview of the many layers of parallelism that are available in modern computer architecture, such as, instruction-level parallelism, data-level parallelism (SIMD), multi-cores, multi-processing). We discuss the properties of these layers and learn how we can effectively use the available parallelism in software.

A key objective of this course is to connect parallel computer architecture and performance-oriented software development both in theory and practice. Hence, the course involves a signficiant amount of programming exercises and practial labs where the code is executed on the HPC systems Noctua 1/2 at Paderborn Center for Parallel Computing.

As programming language, we will use Julia, a modern, high-performance, dynamically typed language tailored for scientific computing. Julia has a very rich ecosystem of extension packages for numerical algorithms, data analysis, machine learning, visualization and highly specialized packages for numerous application domains. On the other side, Julia is well suited as an interactive programming environment, which is ideal for learning and explorative research.

For the practical application of the discussed programming paradigms and optimization techniques we will work with case studies from the domains of:

• Computational physics
• Combinatorial optimization
• Image classification
• Linear algebra
• etc.

Tentative List of Topics

This course is offered for the first time. Hence, the selection of topics is still in flux. The current tentative list of topics is:

1. A tour of parallel and high-performance computer architecture
2. Overview of HPC with Julia
3. Performance Evaluation
4. Performance Optimization of Serial Programs
5. Patterns and Models of Parallel Programming
6. SIMD/Vectorization
7. Multi-Threaded Programming for Concurrency and Performance
8. Distributed Memory Programming
9. Distributed Memory Programming with MPI
10. Deep Learning
11. GPU Programming

Materials

• Lecture slides (available in PANDA)
• Companion website

Further Reading

• Julia website
• Julia for high-performance scientific computing
• Fast Track to Julia cheat sheet