Numerical Analysis for Artifical Intelligence 1000-2M20MSI
1. Review of Programming in Python/NumPy/IPython notebook and Calculus and Linear Algebra topics (4-5 lectures).
2. Gradient descent and convex optimization, backpropagation algorithm, accelerated gradient descent, solving regression using gradient descent (2-3 lectures).
3. Nonconvex optimization: supervised learning of feed-forward Neural Networks: Differences in Convex/Nonconvex optimization, single/multiple hidden layered feed-forward networks, various activation functions and losses, (4-5 lectures).
4. Introduction to policy gradient methods for reinforcement learning (2-3 lectures).
5. Higher-order methods for optimization including (Pseudo-)Newton for finding min/max and TRPO for reinforcement learning (1 - 2 lectures).
Type of course
Prerequisites (description)
Learning outcomes
Knowledge:
• Understands theoretical fundamentals and practical implementations of (non)convex optimization methods.
• Have basic knowledge of machine learning.
• Understands algorithms for the training of neural networks.
Skills:
• Can implement basic numerical methods including PCA and linear/logistic regression.
• Can implement basic neural networks and algorithms for training them utilizing gradient descent from scratch in Python.
Competences:
• Can select appropriate machine learning methods for the given problem.
Assessment criteria
Final grades will be granted solely on the basis of the final project and the course homeworks (computer programs).
Bibliography
• Numerical Analysis for AI course notes (a bit outdated as of now) https://github.com/dzako/NA4AI,
• Carl D. Meyer, Matrix Analysis and Applied Linear Algebra, SIAM,
• Jorge Nocedal, Stephen J. Wright, Numerical Optimization, Springer Series in Operations Research,
• Ian Goodfellow and Yoshua Bengio and Aaron Courville, Deep Learning, http://www.deeplearningbook.org
Additional information
Information on level of this course, year of study and semester when the course unit is delivered, types and amount of class hours - can be found in course structure diagrams of apropriate study programmes. This course is related to the following study programmes:
- Bachelor's degree, first cycle programme, Computer Science
- Master's degree, second cycle programme, Computer Science
Additional information (registration calendar, class conductors, localization and schedules of classes), might be available in the USOSweb system: