High-Performance Computing - High-Performance Computing

High-Performance Computing (2V, 3U)

Lecture number	L.079.05733
Term	Winter term 2017/18
Program	Master's program Computer Science and Computer Engineering
Lecturers	Prof. Dr. Christian Plessl

Goals and Contents of the Lecture

The goal of this course is to teach the fundamentals of high-performance computing. The emphasis of the course is on programming. That is, we will discuss programming models, languages and frameworks for efficiently using parallel computer sytems. The lecture will be complemented by a considerable amount of practical programming exercises that allow the students to gain practical experiences with programming, performance optimization and debugging parallel computer systems. To this end, the student will get access to the HPC clusters operated by the Paderborn Center for Parallel Computing (PC²).

The lecture and exercises will be based on the textbook Peter S. Pacheco, An introduction to Parallel Programming, Morgan Kaufmann publishers, 2011. The book is available online within the Paderborn University Network (use VPN for DFN-AAI for access from outside).

The book comprises a number of code excerpts from programs that illustrate the use of the parallel programming techniques introduced in the book. The source code for these examples is available here.

Changelog

2018-02-09 Added slides on Practical performance engineering
2018-01-23 Updated slides on advanced MPI with minor corrections (see changelog) in slides
2018-01-22 Updated slides on advanced MPI with chapter on hybrid MPI+OpenMP applications
2018-01-16 Added slides on advanced MPI, updated schedule
2018-01-09 Added last minute improvements to OpenMP Addon 1 slides
2018-01-08 Added slides for parallel program development
2018-01-08 Added extended version of OpenMP Addon 1 slides
2018-01-03 Completed lecture schedule until end of term
2017-12-18 Added information on exam dates
2017-12-15 Added schedule for last week before Christmas
2017-12-12 Added additional slides for OpenMP
2017-11-28 Published exercise 5 to repository, updated schedule
2017-11-27 Added slides for OpenMP, updated schedule
2017-11-23 Published exercise 4 to exercises repository
2017-11-21 Added the source code for the examples in the textbook
2017-11-20 Cosmetic updates to slides for MPI and Pthreads
2017-11-20 Room change for exercise on 24 November announced
2017-11-20 Added slides for Pthreads chapter
2017-11-06 Updated lecture and exercise schedule, added source code examples for MPI
2017-10-27 Added hello-mpi.c and cheat sheet for tutorial on getting started with HPC cluster
2017-10-24 The link to the textbook has been replaced, full-text access should work now
2017-10-24 Specified lecture and break timings for Friday appointments
2017-10-23 Added preliminary information on programming exercises and bonus system
2017-10-23 Added slides for MPI, update schedule
2017-10-17 Minor updates to slides Introduction
2017-10-16 Minor updates to slides Introduction and Fundamentals
2017-10-13 Added slides for Foundations
2017-10-10 Minor updated to slides for lecture Organization

Schedule and Material

The lecture and exercises will be held weekly on Tuesday 9:15-10:45 in lecture room O2 and on Friday 11:15-13:45 (30 minute lunch break at 12:15) in lecture room O1.258. Typically, the appointments on Tuesday will be used for the lecture and the Friday appointments will be used for theoretical and practical exercises.

The current schedule is presented in the table below, which will be updated continuously. The following abbreviations will be used in the table below:

L: lecture
S: Supplemental material
H: handout of exercises
D: discussion of exercises
P: practical work on programming exercises (with lecturer or tutor presents for support)

Schedule High-Performance Computing
Date	Contents and Materials
Oct 10, 2017	L: Lecture organization L: Introduction H: Exercises Introduction (Pacheco Ex 1.1, 1.3, 1.4, 1.5)
Oct 13, 2017	No exercise/lecture. Exercises will start on October 20.
Oct 17, 2017	L: Introduction (cntd) L: Foundations H: Exercises Foundations (Pacheco Ex 2.3, 2.5, 2.10, 2.13, 2.19)
Oct 20, 2017	D: Exercises Introduction
Oct 24, 2017	L: Message Passing Interface (MPI) S: Pacheco Chapter 3 S: LLNL MPI Tutorial
Oct 27, 2017	P: Getting started with HPC Systems at PC² (bring your laptop) H: OCuLUS brief instructions L: Introduction HPC Clusters at PC² P: hello-mpi.c P: hello-mpi-cheatsheet.txt
Oct 31, 2017	public holiday, no lecture
Nov 3, 2017	P: Programming Exercise 1 (MPI Bandwidth)
Nov 7, 2017	L: Message Passing Interface (MPI) (ctnd) H: Exercise 3 S: vector_add.c mpi_vector_add.c S: mat_vect_mult.c mpi_mat_vect_mult.c
Nov 10, 2017	P: Deadline submission Exercise 1 (MPI Bandwidth) P: Programming Exercise 2 (Pi, Butterfly Sum) P: Programming Exercise 3 (Conway Game of Life)
Nov 14, 2017	No lecture
Nov 17, 2017	P: Deadline submission Programming Exercise 2 (Pi, Butterfly Sum) P: Erik Messerli will be there to help you with your programming exercises
Nov 21, 2017	L: Shared memory programming with Pthreads S: Pacheco Chapter 4
Nov 24, 2017	Room change: exercise will be held in P1.6.11 L: Shared memory programming with Pthreads (ctnd) P: Deadline submission Programming Exercise 3 (Conway Game of Life) P: Programming Exercise 4 (Pthreads) D: Programming Exercise 1
Nov 28, 2017	L: Shared Memory Programming With OpenMP S: Pacheco Chapter 5
Dec 1, 2017	P: Submission deadline for Exercise 4 P: Erik Messerli will be there to help you with your programming exercises
Dec 5, 2017	No lecture
Dec 8, 2017	No exercise
Dec 12, 2017	L: Shared Memory Programming with OpenMP (ctnd) L: Additional slides for OpenMP (Addon 1)
Dec 15, 2017	P: Submission deadline for Exercise 5
Dec 19, 2017	Christmas lecture: we will do a guided tour through our HPC center and look at the server rooms, cooling and energy supply infrastructure.
Dec 22, 2017	Cancelled
Jan 9, 2018	L: Parallel Program Development (Case Studies) S: Pacheco Chapter 6
Jan 12, 2018	D: Discussion of the solution to Exercises 2, 3 and 4
Jan 16, 2018	L: Parallel Program Development (cntd) L: Advanced MPI (derived data types, non-blocking communication, one-sided communication, hybrid MPI/OpenMP programming)
Jan 19, 2018	L: Advanced MPI (one-sided communication) P: Submission deadline for Exercise 6 H: Programming Exercise 7 (Mandelbrot, MPI RMA)
Jan 23, 2018	L: Advanced MPI (ctnd) L: Lecture evaluation (VKrit) L: Collection of questions for Q&A session
Jan 26, 2018	D: Discussion of Exercises 5 + 6 P: Submission of Exercise 7 (Mandelbrot)
Jan 30, 2018	L: Practical performance engineering
Feb 2, 2018	L: Practical performance engineering (ctnd) P: Submission of Exercise 8 (due on Feb 1!) L: Q&A session L: Wrapup, discussion of exam modalities, feedback

Programming Exercises

As this course is supposed to teach practical skills in HPC as well, there will be programming exercises to all topics covered in the lectured. The exercises comprise:

Basic programming exercises taken from the book of Pacheo. These exercises lay the practical foundations on the concepts discussed in class. Solving one of these exercises typically takes less than one hour.
A set of 3 programming projects that are more challenging and hopefully also more entertaining. Solving these challenges is expected to take several hours.

To reward participation in the programming exercises the contents of the exercises are also relevant and part of the exam. Also, students who submit solutions to the programming exercises (basic and projects) can earn bonus points for the exam. There will be three levels of bonus points that will result in an increase of the grade by 0.3, 0.7, or 1.0. The precondition for the bonus to be applied is that the exam without considering the bonus must sufficient for passing (at least 4.0).

To assign the bonus points, each exercise submission is evaluated and rated. Each submission must comprise a brief report describing the results, findings, or relevant implementation details. The rating of exercises does not only consider correct solutions but reasonable attempts to solve the problems are also rewarded. In this case, I expect that you describe the problems and how you attempted to solve the problems in a systematic way as part of the report.

Achieving at least 25% of the points results in a bonus of 0.3; achieving at least 50% in a bonus of 0.7, and achieving more than 75% in a bonus of 1.0. The deadline for submitting the solutions to the programming exercises is stated in the lecture schedule on the website and announced in the lecture.

Since you can earn a substantial bonus in the exercises, you are expected to solve them in groups of at most two students.

You find a list of all programming exercises, contents, source code and deadline information in the GitLab repository for this course.

The submission of the solutions will happen through Koala. Please upload a ZIP file with the source code to your solution and a brief report or README file document your results.

Exam

The exam will be an oral exam of approximately 40 minutes. All topics covered in the lecture (slides, classroom examples) and exercises.

The appointment for exams will be scheduled in one of the following exam time slots.

9 February 2018, 9:00-16:00
16 March 2018, 9:00-16:00
further time slots in April or later will be assigned on deman.

For scheduling an exam, please send your request by email to my administrative assistant Mrs. Kemper.