Parallel programming 4010-PRa

1. Introduction to Parallel Programming - Moore's Law, supercomputers, Flynn's taxonomy (2 hours, lecture).
2. Introduction to Parallel Programming - memory access, basic concepts (2 hours, lecture).
3. Introduction to Parallel Programming - runtime, speedup, Amdahl Law, Gustafson's Law, communication, synchronization (2 hours, lecture).
4. Introduction to Parallel Programming - decomposition, load balancing, parallel algorithm design methods, object-oriented programming Models, Programming Patterns (2 hours, lecture).
5. OpenMP - HelloWorld in C with and without OpenMP, compiling and running C programs with and without OpenMP, the parallel construct (2 hours, lab).
6. OpenMP - controlling the number of threads, heap dynamic variables (2 hours, lecture).
7. OpenMP - C functions, private, shared, and firstprivate variables, the reduction operator (2 hours, lecture).
8. OpenMP - two-dimensional arrays (stack, data area, heap), various time measurement methods (2 hours, lecture).
9. OpenMP - C functions, private, shared, and firstprivate variables, reduction operator, two-dimensional arrays (2 hours, lab).
10. OpenMP - parallel loops, non-iterative division of labor (2 hours, lecture).
11. OpenMP - parallel loops, timing, non-iterative division of labor (2 hours, lab).
12. OpenMP - thread coordination, race condition, deadlock, false sharing, thread safety, best practices (2 hours, lecture).
13. OpenMP - stencil computation (6 hours, lab).
14. MPI - introduction, hello world, data types in MPI (2 hours, lecture).
15. MPI - point-to-point blocking communication (2 hours, lab).
16. MPI - hello world, point-to-point blocking communication (2 hours, lab).
17. MPI - non-blocking communication, Timing (2 hours, lecture).
18. MPI - non-blocking communication (2 hours, lab).
19. MPI - implementing the client-server pattern (2 hours, lab).
20. MPI - Null Mpi process, collective operations (2 hours, lecture).
21. MPI - non-circular shift, collective operations, and file handling: implementing parallel text search in a text file (4 hours, lab).
22. PCJ - introduction, Hello World, Execution Builder (2 hours, lecture).
23. PCJ - Hello World with PCJ - compiling and running the program (2 hours, lab).
24. PCJ - methods for reporting the number of tasks and nodes, shared variables (2 hours, lecture).
25. PCJ - generating a random matrix, calculating row sums, using properties to pass call arguments (2 hours, lab).
26. PCJ - synchronous and asynchronous one-sided and collective operations (2 hours, lecture).
27. PCJ - implementing all-broadcast and all-reduce (2 hours, lab).

The order of topics covered and the level of detail may vary slightly.