Statistics 1000-116bST

The course is an introduction to classical mathematical statistics. Below are the concepts and results that will be covered during the lectures. The material in square brackets is optional and will be presented if time permits.

- Sample counterparts of population quantities: mean, variance, standard deviation, quantiles. Empirical distribution function, Glivenko-Cantelli theorem. Empirical distribution.

- Statistical models: nonparametric, semiparametric, parametric.
Sufficient and minimal sufficient statistics. Factorization criterion and Dynkin-Lehmann-Scheffé theorem. Complete statistics.

- Exponential families and their properties.

- Parameter estimation: method of relative frequency, method of moments, maximum likelihood method. [EM algorithm]

- Bias of an estimator and mean squared error.

- Minimum variance unbiased estimators. Rao-Blackwell theorem. [Lehmann-Scheffé theorem]

- Fisher information, Cramer-Rao information inequality.
Asymptotic properties of estimators. Consistency, "Delta method". Asymptotic normality of the maximum likelihood estimator.

- Confidence intervals.

- Hypothesis testing theory: significance level, power, p-value. Uniformly most powerful test, Neyman-Pearson lemma. [Karlin-Rubin theorem]. Likelihood ratio test. [Wald test]

- Chi-squared test for simple and composite hypotheses. Chi-squared test of independence as a special case. Kolmogorov-Smirnov test, two-sample t-test. [Nonparametric tests: runs test, Fisher’s exact test]

- Gaussian linear models. Maximum likelihood estimators for coefficients, confidence intervals, hypothesis testing of zero coefficients. ANOVA.

- [Ridge regression and LASSO, logistic regression]

- [Elements of Bayesian statistics]

The accompanying computer lab sessions aim to illustrate the concepts introduced in the lecture and teach basic data analysis using the R programming language.