Revealing statistical uncertainties hidden in reserch results 1400-237UNSUWB-OG

The course introduces selected statistical methods in an atypical way: most part of exercises consist in finding confidence limits in situations when no raw data are available, but only their summaries and results of statistical significance testing - which is a prevailing standard. The focus thus shifts from statistical significance to quantitative consideration of effects and uncertainties about them. This prevents from succumbing to the charms of significance and false certainty it provides. We rather ask what conclusions of subject-matter importance can be drawn about statistical populations and with how high confidence. The first illustrative example is a two-groups comparison: given sample means and p-value from the common Student's t-test we try to establish how big (at least, and at most) is the true value of effect. This can be achieved by a kind of reverse-enineering. Reverse-engineering exercises of such sort form a large part of the course. They require quite a great deal of going into the details of the methods considered and thus lead to close acquaintance with them.

Such an approach is applied to diverse problems and methods: comparisons of means, frequencies, associations, analysis of variance, correlation, regression etc. When no exact solutions exist, we look for reasonable approximations. In the case of two-way ANOVA, careful attention is paid to the analysis of interactions. When applicable, permutation (re-randomization) and bootstrap methods are taken into account. Computations are conducted mostly in R and SAS, but some programs dedicated to aspects of statistical uncertainties available in the internet are also employed. Although stylized examples are used in most cases, they bear high resemblance to situations commonly encountered in scientific literature.

To supplement a picture, Monte Carlo simulations are performed, which show variability of the results of, and conclusions from, repetitions of the same experiment. They can provide insights into statistical uncertainties in more complex cases, when no methods exist allowing a transition from statistical significance to meaningful uncertainty intervals.

General view on statistical uncertainties in research, without considering computational details, is the subject of a separate lecture - "The cult of statistical significance".

Main fields of studies for MISMaP

environmental protection
geology
mathematics
astronomy
geography
biotechnology
spatial development
psychology
physics
computer science
biology
chemistry
applied geology

Type of course

general courses
elective courses
optional courses
elective monographs

Mode

Classroom

Prerequisites (description)

Participants are expected to have some basic knowledge about statistical inference and have no fear of simple mathematical formulae. Some elementary experience in programming is useful, but not absolutely necessary.

Course coordinators

Tomasz Wyszomirski

Bibliography

Literature (additions to be made during the course):

Bird KD. 2004. Analysis of Variance via Confidence Intervals. SAGE.

Cumming G. 2011. Understanding the New Statistics: Effect Sizes, Confidence Intervals, and Meta-Analysis. Routledge.

Halsey LG, Curran-Everett D., Vowler SL, Drummond G. 2015. The fickle P value generates irreproducible results. Nature Methods, 12: 179-185.

Ioannidis J.P.A. 2005. Why most published research findings are false. PLoS Med 2(8): e124.

Kline R.B. 2004. Beyond Significance Testing. Reforming Data Analysis Methods in Behavioral Research. American Psychological Association.

Krishnamoorthy K., Mathew T. 2009. Statistical Tolerance Regions. Theory, Applications, and Computation. Wiley.

Lazzeroni LC, Lu Y, Belitskaya-Levy I. 2014. P-values in genomics: Apparent precision masks high uncertainty. Molecular Psychiatry, 19: 1336–1340.

Lecoutre B, Poitvineau J. 2014. The Significance Test Controversy Revisited. The Fiducial Bayesian Alternative. Springer.

Meeker WQ, Hahn GJ, Escobar LA. 2017. Statistical intervals: A guide for practitioners and researchers. Wiley.

Motulsky H. 2014. Intuitive Biostatistics, 3rd edition. Oxford University Press.

Nuzzo R. 2014. Scientific method: statistical errors. Nature 506: 150-152.

Schweder T, Hjort NL. 2016. Confidence, Likelihood, Probability: Statistical Inference with Confidence Distributions. Cambridge University Press.

Wang C. 1992. Sense and Nonsense of Statistical Inference: Controversy, Misuse, and Subtlety. CRC Press.

Additional information

Information on level of this course, year of study and semester when the course unit is delivered, types and amount of class hours - can be found in course structure diagrams of apropriate study programmes. This course is related to the following study programmes:

Additional information (registration calendar, class conductors, localization and schedules of classes), might be available in the USOSweb system:

Description of 1400-237UNSUWB-OG in USOSweb