Statistical machine learning 1000-2M21US

The detailed program

1. 
   Explanatory Statistics (2-3 lessons)
   




1. 
   Basic summary statistic (sample mean, median, sample variance, etc.)
   


2. 
   Visualization of data (histogram, box-plots, kernel density estimation)
   


3. 
   Principal component analysis
   


4. 
   Clusterization, hierarchical clustering, k-means, k-medoids 
   




2. 
   Statistical theory (4-5 lessons)
   




1. 
   Basic definitions (statistical models, statistics, likelihood, etc.)
   


2. 
   Estimation theory (maximum likelihood estimators, efficiency, mean square error, bias vs variance trade-off, confidence intervals)
   


3. 
   Statistical hypothesis testing (type I and type II errors, power of a test, significance, p-value)
   


4. 
   Problems with p-values (effect size, multiple hypotheses testing)
   


5. 
   Bayesian inference (prior distribution, posterior distribution, Bayesian risk and Bayesian estimator, credible intervals)
   


6. 
   Distance between probability measures (Kullback-Leibler divergence, total variation distance, etc.)
   




3. 
   Simple regression and classification models (3-4 lessons)
   




1. 
   Linear regression
   


2. 
   Classification. Logistic regression, LDA, QDA
   


3. 
   Cross-validation and bootstrap
   


4. 
   Model selection and regularization. Lasso, ridge regression, forward-backwards procedure
   




4. 
   Advanced models (3 lessons)
   




1. 
   Tree-like models, bagging, random forests, boosting
   


2. 
   Support vector machines
   


3. 
   Non-linear models: splines, generalized additive models