In­tro­duc­tion to High-Per­form­ance Com­put­ing (BSc CS)

Advance Information

This is an overview of the contents of the lecture "Introduction to High-Performance Computing" that will be offered for the first time in Summer Term 2023.

Course Information

  • Title: High-Performance Computing
  • Lecturer: Prof. Dr. Christian Plessl
  • Lecture Number: L.079.05505
  • Teaching Language: German (but all materials provided in English)

Format

  • 2h lecture
  • 1h theoretical exercise
  • 2h practical exercises/lab per week

Times

  • Lecture: TBD
  • Exercise, lab: TBD

Content Overview

This course aims at providing an Introduction to High-Performance Computing. We will start out with an overview of the many layers of parallelism that are available in modern computer architecture, such as, instruction-level parallelism, data-level parallelism (SIMD), multi-cores, multi-processing). We discuss the properties of these layers and learn how we can effectively use the available parallelism in software.

A key objective of this course is to connect parallel computer architecture and performance-oriented software development both in theory and practice. Hence, the course involves a signficiant amount of programming exercises and practial labs where the code is executed on the HPC systems Noctua 1/2 at Paderborn Center for Parallel Computing.

As programming language, we will use Julia, a modern, high-performance, dynamically typed language tailored for scientific computing. Julia has a very rich ecosystem of extension packages for numerical algorithms, data analysis, machine learning, visualization and highly specialized packages for numerous application domains. On the other side, Julia is well suited as an interactive programming environment, which is ideal for learning and explorative research.

For the practical application of the discussed programming paradigms and optimization techniques we will work with case studies from the domains of:

  • Computational physics
  • Combinatorial optimization
  • Image classification
  • Linear algebra
  • etc.

Tentative List of Topics

This course is offered for the first time. Hence, the selection of topics is still in flux. The current tentative list of topics is:

  1. A tour of parallel and high-performance computer architecture
  2. Overview of HPC with Julia
  3. Performance Evaluation
  4. Performance Optimization of Serial Programs
  5. Patterns and Models of Parallel Programming
  6. SIMD/Vectorization
  7. Multi-Threaded Programming for Concurrency and Performance
  8. Distributed Memory Programming
  9. Distributed Memory Programming with MPI
  10. Deep Learning
  11. GPU Programming

Materials

  • Lecture slides
  • Companion website

Further Reading