(in Polish) Statystyczne uczenie maszynowe 1000-1M18SUM

In recent years, the view has been clarified that deep neural networks are effective where there is a lot of data, the predictors have a certain spatial or temporal organization (such as signals, texts or images) and the signal-to-noise ratio is high. However, where: there is little data or no special organization, or the signal is weak or the results should be interpretable, modern linear methods, such as lasso or gradient boosting, are better (Robert Tibshirani in his presentation after receiving the ISI award, May 2021). The lecture is an introduction to the supervised learning, in other words statistical prediction, focused on such modern methods and partly based on "The Elements of Statistical Learning" by Hastie, Tibshirani and Friedman.

In the first part, I discuss the basic prediction method of the continuous response, that is the linear regression model, and ridge regression, lasso and the best subset selection. In the second part, I present linear prediction methods of the discrete response, that is classifications such as Fisher's linear discriminant analysis, logistic regression, or support vector machines. In the third part, I discuss universal, non-linear predictors, such as the k-nearest neighbors method and decision trees. The fourth part is devoted to the regularization of the learning and boosting the power of prediction. In particular, I discuss penalization of the prediction error, kernelization of explanatory variables and boosting (linear combinations of "weak" predictors). In the last part I will discuss the use of linear methods and deep neural networks (ConvNets, Visual Transformers) for the prediction of image properties such as classification or segmentation.

The lecture is focused on important machine learning methods, which are solutions of the penalized loss minimization on the training data. In this way, we get sets of predictors indexed by a "hyperparameter" (e.g. the weight of the function which penalizes the predictor parameters), which value is calculated on additional validation data or by means of cross validation on the training data. I will devote a lot of attention to rigorously explain popular validation procedures (also called the model selection).