High-Per­for­mance Com­pu­ting (2V, 3U)

 

 

Lecture number L.079.05733
Term Winter term 2017/18
Program Master's program Computer Science and Computer Engineering
Lecturers Prof. Dr. Christian Plessl

Goals and Contents of the Lecture

The goal of this course is to teach the fundamentals of high-performance computing. The emphasis of the course is on programming. That is, we will discuss programming models, languages and frameworks for efficiently using parallel computer sytems. The lecture will be complemented by a considerable amount of practical programming exercises that allow the students to gain practical experiences with programming, performance optimization and debugging parallel computer systems. To this end, the student will get access to the HPC clusters operated by the Paderborn Center for Parallel Computing (PC²).

The lecture and exercises will be based on the textbook Peter S. Pacheco, An introduction to Parallel Programming, Morgan Kaufmann publishers, 2011. The book is available online within the Paderborn University Network (use VPN for DFN-AAI for access from outside).

The book comprises a number of code excerpts from programs that illustrate the use of the parallel programming techniques introduced in the book. The source code for these examples is available here.

Chan­ge­log

  • 2018-02-09 Added slides on Practical performance engineering
  • 2018-01-23 Updated slides on advanced MPI with minor corrections (see changelog) in slides
  • 2018-01-22 Updated slides on advanced MPI with chapter on hybrid MPI+OpenMP applications
  • 2018-01-16 Added slides on advanced MPI, updated schedule
  • 2018-01-09 Added last minute improvements to OpenMP Addon 1 slides
  • 2018-01-08 Added slides for parallel program development
  • 2018-01-08 Added extended version of OpenMP Addon 1 slides
  • 2018-01-03 Completed lecture schedule until end of term
  • 2017-12-18 Added information on exam dates
  • 2017-12-15 Added schedule for last week before Christmas
  • 2017-12-12 Added additional slides for OpenMP
  • 2017-11-28 Published exercise 5 to repository, updated schedule
  • 2017-11-27 Added slides for OpenMP, updated schedule
  • 2017-11-23 Published exercise 4 to exercises repository
  • 2017-11-21 Added the source code for the examples in the textbook
  • 2017-11-20 Cosmetic updates to slides for MPI and Pthreads
  • 2017-11-20 Room change for exercise on 24 November announced
  • 2017-11-20 Added slides for Pthreads chapter
  • 2017-11-06 Updated lecture and exercise schedule, added source code examples for MPI
  • 2017-10-27 Added hello-mpi.c and cheat sheet for tutorial on getting started with HPC cluster
  • 2017-10-24 The link to the textbook has been replaced, full-text access should work now
  • 2017-10-24 Specified lecture and break timings for Friday appointments
  • 2017-10-23 Added preliminary information on programming exercises and bonus system
  • 2017-10-23 Added slides for MPI, update schedule
  • 2017-10-17 Minor updates to slides Introduction
  • 2017-10-16 Minor updates to slides Introduction and Fundamentals
  • 2017-10-13 Added slides for Foundations
  • 2017-10-10 Minor updated to slides for lecture Organization

Sche­du­le and Ma­te­ri­al

The lecture and exercises will be held weekly on Tuesday 9:15-10:45 in lecture room O2 and on Friday 11:15-13:45 (30 minute lunch break at 12:15) in lecture room O1.258. Typically, the appointments on Tuesday will be used for the lecture and the Friday appointments will be used for theoretical and practical exercises.

The current schedule is presented in the table below, which will be updated continuously. The following abbreviations will be used in the table below:

  • L: lecture
  • S: Supplemental material
  • H: handout of exercises
  • D: discussion of exercises
  • P: practical work on programming exercises (with lecturer or tutor presents for support)
Schedule High-Performance Computing
Date Contents and Materials
Oct 10, 2017
Oct 13, 2017
  • No exercise/lecture. Exercises will start on October 20.
Oct 17, 2017
Oct 20, 2017
  • D: Exercises Introduction
Oct 24, 2017
Oct 27, 2017
Oct 31, 2017
  • public holiday, no lecture
Nov 3, 2017
Nov 7, 2017
Nov 10, 2017
Nov 14, 2017
  • No lecture
Nov 17, 2017
  • P: Deadline submission Programming Exercise 2 (Pi, Butterfly Sum)
  • P: Erik Messerli will be there to help you with your programming exercises
Nov 21, 2017
Nov 24, 2017
  • Room change: exercise will be held in P1.6.11
  • L: Shared memory programming with Pthreads (ctnd)
  • P: Deadline submission Programming Exercise 3 (Conway Game of Life)
  • P: Programming Exercise 4 (Pthreads)
  • D: Programming Exercise 1
Nov 28, 2017
Dec 1, 2017
  • P: Submission deadline for Exercise 4
  • P: Erik Messerli will be there to help you with your programming exercises
Dec 5, 2017
  • No lecture
Dec 8, 2017
  • No exercise
Dec 12, 2017
Dec 15, 2017
  • P: Submission deadline for Exercise 5
Dec 19, 2017
  • Christmas lecture: we will do a guided tour through our HPC center and look at the server rooms, cooling and energy supply infrastructure. 
Dec 22, 2017
  • Cancelled
Jan 9, 2018
Jan 12, 2018
  • D: Discussion of the solution to Exercises 2, 3 and 4
Jan 16, 2018
  • L: Parallel Program Development (cntd)
  • L: Advanced MPI (derived data types, non-blocking communication, one-sided communication, hybrid MPI/OpenMP programming)
Jan 19, 2018
  • L: Advanced MPI (one-sided communication)
  • P: Submission deadline for Exercise 6
  • H: Programming Exercise 7 (Mandelbrot, MPI RMA)
Jan 23, 2018
  • L: Advanced MPI (ctnd)
  • L: Lecture evaluation (VKrit)
  • L: Collection of questions for Q&A session
Jan 26, 2018
  • D: Discussion of Exercises 5 + 6
  • P: Submission of Exercise 7 (Mandelbrot)
Jan 30, 2018
Feb 2, 2018
  • L: Practical performance engineering (ctnd)
  • P: Submission of Exercise 8 (due on Feb 1!)
  • L: Q&A session
  • L: Wrapup, discussion of exam modalities, feedback

Pro­gram­ming Exer­ci­ses

As this course is supposed to teach practical skills in HPC as well, there will be programming exercises to all topics covered in the lectured. The exercises comprise:

  1. Basic programming exercises taken from the book of Pacheo. These exercises lay the practical foundations on the concepts discussed in class. Solving one of these exercises typically takes less than one hour.
  2. A set of 3 programming projects that are more challenging and hopefully also more entertaining. Solving these challenges is expected to take several hours.

To reward participation in the programming exercises the contents of the exercises are also relevant and part of the  exam. Also, students who submit solutions to the programming exercises (basic and projects) can earn bonus points for the exam. There will be three levels of bonus points that will result in an increase of the grade by 0.3, 0.7, or 1.0. The precondition for the bonus to be applied is that the exam without considering the bonus must sufficient for passing (at least 4.0).

To assign the bonus points, each exercise submission is evaluated and rated. Each submission must comprise a brief report describing the results, findings, or relevant implementation details. The rating of exercises does not only consider correct solutions but reasonable attempts to solve the problems are also rewarded. In this case, I expect that you describe the problems and how you attempted to solve the problems in a systematic way as part of the report.

Achieving at least 25% of the points results in a bonus of 0.3; achieving at least 50% in a bonus of 0.7, and achieving more than 75% in a bonus of 1.0. The deadline for submitting the solutions to the programming exercises is stated in the lecture schedule on the website and announced in the lecture.

Since you can earn a substantial bonus in the exercises, you are expected to solve them in groups of at most two students.

You find a list of all programming exercises, contents, source code and deadline information in the GitLab repository for this course.

The submission of the solutions will happen through Koala. Please upload a ZIP file with the source code to your solution and a brief report or README file document your results.

Ex­am

The exam will be an oral exam of approximately 40 minutes. All topics covered in the lecture (slides, classroom examples) and exercises.

The appointment for exams will be scheduled in one of the following exam time slots.

  • 9 February 2018, 9:00-16:00
  • 16 March 2018, 9:00-16:00
  • further time slots in April or later will be assigned on deman. 

For scheduling an exam, please send your request by email to my administrative assistant Mrs. Kemper.