Parallel Programming

Module overview

This module looks at the various types of parallel compute devices available in modern systems, and how they can be used to either improve performance, reduce energy consumption, or both. The types of devices considered include multi-core CPUs, many-core CPUs, SIMD extensions, GPUs, NPUs/TPUs, and other types of evolving compute hardware. The particular focus of this course is on how to program and optimise for such devices, including analysing workloads for bottlenecks, choosing or creating parallel algorithms, and then profiling and testing. In most cases there will also be multiple types of parallelism applied at once, using combinations of threaded, SIMD, pipelines, and task-level parallelism. After completing the module students should be able to tackle the acceleration of quite complex programs, and realistically achieving speedups of 10x .. 100x over baseline implementations.

Aims and Objectives

Learning Outcomes

Knowledge and Understanding

Having successfully completed this module, you will be able to demonstrate knowledge and understanding of:

How to capture parallelism using the concept of iteration spaces and tasks
The types of parallelism provided by contemporary hardware platforms
How parallelism can speed-up programs, and why this speed-up is often limited

Subject Specific Intellectual and Research Skills

Having successfully completed this module you will be able to:

Analyse a given program and identify opportunities for parallelism
Identify data-races in a program and use locking or synchronisation to remove them
Use mathematical abstractions and techniques to analyse performance opportunities and identify bottlenecks

Subject Specific Practical Skills

Having successfully completed this module you will be able to:

Use CUDA or OpenCL to accelerate a given function using a GPU
Accelerate a given application in a general-purpose CPU using threads, SIMD, and memory optimisations

Learning and Teaching

Teaching and learning methods

Lectures, problem-classes, and independent coursework

Study time
Type	Hours
Revision	10
Problem Classes	12
Guided independent study	46
Lecture	36
Completion of assessment task	46
Total study time	150

Assessment

Summative

This is how we’ll formally assess what you have learned in this module.

Breakdown
Method	Percentage contribution
Computing assignment	30%
Computing assignment	30%
Exam	40%

Referral

This is how we’ll assess you if you don’t meet the criteria to pass this module.

Breakdown
Method	Percentage contribution
Exam	100%

Repeat

An internal repeat is where you take all of your modules again, including any you passed. An external repeat is where you only re-take the modules you failed.

Breakdown
Method	Percentage contribution
Exam	100%