(in Polish) High-Performance Computing for Artificial Intelligence 2 4010-HPAI

Artificial Intelligence (AI) has become a widely used tool in scientific work. Not only does AI enable advanced data analysis and visualization but it can also replace traditional numerical simulations or speed them up by making accurate predictions of interactions in a system. Additionally, AI allows for quick searching of scientific publication databases and generating article summaries. It is also being used more frequently for the full writing of scientific papers. However, the development of more sophisticated AI models requires the processing of huge amounts of data and the costly, time-consuming process of training AI models. This is especially evident with large language models (LLMs), where the training process can take weeks on thousands of the most powerful GPUs. Working with such complex models requires the use of large computing power: computing clusters or a cloud, which is the domain of High-Performance Computing (HPC).
HPC is a field of computer science dedicated to solving the most demanding computational problems in science. Top well-known applications include: (i) predicting the properties of semiconductors using quantum mechanical calculations with Density Functional Theory (DFT), (ii) drug design through molecular dynamics simulations, and (iii) numerical weather forecasting. Such calculations require hundreds or even thousands of CPUs. The key element of HPC is parallel computing. It allows calculations to be sped up by using many CPUs or GPUs. It also helps reduce the memory needed for each single computing core. For many years, parallel computing has been essential for scientific calculations, and now it is a crucial technique for AI. This is why the techniques, methodologies, and algorithms developed for HPC tasks are now also being applied to the training of AI models.

This lecture will focus on three main aspects:
1. What HPC is, how to work with computing clusters, and how parallel computing can be used in scientific research?
2. How to speed up Python code execution, with particular focus on mathematical operations, and how to parallelize these operations or use computational accelerators (like GPUs).
3. How to improve the efficiency of AI model training and apply parallel programming techniques to AI model training.

The session will be conducted in a workshop format, allowing participants to gain both theoretical knowledge and practical skills. Moreover, practical sessions will reinforce the acquired theoretical knowledge and provide hands-on experience. The exercises will be conducted on an HPC computer used by leading research teams in Poland.