Understanding Deep Learning 2400-ZEWW948

Module 1: Introduction & Supervised Learning (6 hours)
- What is deep learning? History and applications. (1 hour)
- Supervised learning: Regression and classification. (2 hours)
- Perceptron and Multi-Layer Perceptron (MLP). (3 hours)
Module 2: Shallow & Deep Neural Networks (6 hours)
- Activation functions, backpropagation, and gradient descent. (2 hours)
- Building and training MLPs for classification and regression. (2 hours)
- Introduction to deep neural networks and their advantages. (2 hours)
Module 3: Loss Functions, Fitting & Performance (6 hours)
- Different types of loss functions and choosing the right one. (2 hours)
- Model selection, evaluation metrics, and cross-validation. (2 hours)
- Overfitting, underfitting, and regularization techniques. (2 hours)
Module 4: Advanced Architectures (9 hours)
- Convolutional Neural Networks (CNNs) for image recognition. (2 hours)
- Kolmogorov-Arnold Networks: Universal approximation and applications. (2 hours)
- Generative Adversarial Networks (GANs): Architecture, training, and applications in image generation. (2 hours)
- Transformers and LLMs: Attention mechanism and language modelling. (3 hours)
Module 5: Computing Gradients & Initialization (3 hours)
- Automatic differentiation and backpropagation in detail. (1 hours)
- Gradient-based optimization algorithms (SGD, Adam, RMSprop). (1 hours)
- Initialization strategies for deep networks and vanishing/exploding gradients. (1 hours)