Reproducibility and transparency in management and organization research 1600-SZD-N-RTWB-ZJ
This course examines the concepts of reproducibility and transparency in the context of scientific research in management and organization. Students will gain a comprehensive understanding of the challenges and best practices associated with ensuring the reliability and validity of research findings. Through a combination of lectures, discussions, case studies, and hands-on exercises, students will develop the skills necessary to critically evaluate and conduct research that promotes transparency and rigor in the field.
Block 1: Introduction to Reproducibility, Replicability, and Transparency (2 hours)
The great reckoning: precognition, Open Science Collaboration, and the dishonest research on dishonesty.
Understanding the importance of reproducibility, replicability, and transparency in scientific research.
Differentiating between reproducibility and replicability
Historical context and common challenges in research transparency
Block 2: Methodological Foundations (2 hours)
Sources of bias and how to mitigate them: The garden of forking paths, p-hacking, and HARK-ing.
Research design and its impact on reproducibility and replicability
Sampling techniques and generalizability of findings
Validity and reliability of measurements
Transparency in data collection and analysis
Block 3: Enhancing Reproducibility and Replicability (2 hours)
Pre-registration and registered reports
Open science practices and data sharing
Version control and documentation
Replication studies and meta-analyses
Tools and technologies for reproducible research
Block 4: Ensuring Data Integrity and Transparency (2 hours)
Data management and organization for reproducibility
Data cleaning, preprocessing, and transformation
Dealing with outliers and influential observations
Ensuring data integrity and accuracy
Documentation and annotation of data sources and variables
Block 5: Implementing Reproducible Practices (2 hours)
Building a culture of reproducibility in organizations and academic institutions
The role of journals, funding agencies, and professional associations in promoting reproducibility
Case studies and real-world examples of successful reproducible research projects
Practical exercises and tools for implementing reproducible practices
Type of course
Course coordinators
Learning outcomes
Knowledge: Knows and understands:
WG_1 - to the extent necessary for existing paradigms to be revised - a worldwide body of work, covering theoretical foundations as well as general and selected specific issues - relevant to a particular discipline within the social sciences
WG_2 - the main development trends in the disciplines of the social sciences in which the education is provided
WG_3 - scientific research methodology in the field of the social sciences
WG_4 - the principles for dissemination of scientific results in the field of the social sciences, including in the open access mode
WK_1 - fundamental dilemmas of modern civilisation from the perspective of the social sciences
WK_2 - economic, legal, ethical and other relevant determinants of scientific activity in the field of the social sciences
Skills: Can
UW_1 - make use of knowledge from various fields of science, in particular the social sciences in order to creatively identify, formulate and innovatively solve complex problems or perform tasks of a research nature, and in particular to: - define the purpose and object of scientific research in the field of the social sciences, formulate a research hypothesis, - develop research methods, techniques and tools and apply them creatively, - make inferences based on scientific findings
UW_2 - mcritically analysing and evaluating the results of scientific research in the field of the social sciences, expert and other creative work and their contribution to the advancement of knowledge.
Social competences: Is ready to:
KK_1 - critically evaluating achievements within a given scientific discipline in the field of the social sciences
KK_2 - critically assessing one's own contribution to the development of a scientific discipline
KK_3 - recognising the importance of knowledge in solving cognitive and practical problems within a specific discipline in the field of the social sciences
Assessment criteria
description of requirements related to participation in classes, including the
permitted number of explained absences; One absence is permitted.
principles for passing the classes and the subject (including resit session); Participants are required to prepare their own transparency and reproducibility plan for their own research.
methods for the verification of learning outcomes; Written credit (Project work: preparation of a plan to ensure the reproducibility and transparency of the research).
evaluation criteria: The reproducibility and transparency plan will be evaluated against the principles discussed during the course.
Practical placement
-
Bibliography
Aguinis, H., Ramani, R. S., & Alabduljader, N. (2018). What You See is What You Get? Enhancing Methodological Transparency in Management Research. Academy of Management Annals, 83.
Camerer, C. F., Dreber, A., Forsell, E., Ho, T.-H., Huber, J., Johannesson, M., . . . Wu, H. (2016). Evaluating replicability of laboratory experiments in economics. Science, 351(6280), 1433-1436. doi:10.1126/science.aaf0918
Camerer, C. F., Dreber, A., Holzmeister, F., Ho, T.-H., Huber, J., Johannesson, M., . . . Wu, H. (2018). Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015. Nature Human Behaviour, 2(9), 637-644. doi:10.1038/s41562-018-0399-z
Ethiraj, S. K., Gambardella, A., & Helfat, C. E. (2016). Replication in Strategic Management. Strategic Management Journal, 37(11), 2191-2192. doi:10.1002/smj.2581
Hensel, P. G. (2019). Supporting replication research in management journals: Qualitative analysis of editorials published between 1970 and 2015. European Management Journal, 37(1), 45-57. doi:10.1016/j.emj.2018.03.004
Hensel, P. G. (2021). Reproducibility and replicability crisis: How management compares to psychology and economics – A systematic review of literature. European Management Journal, 39(5), 577-594. doi:10.1016/j.emj.2021.01.002
Hensel, P. G. (2023a). Dissecting the tension of open science standards implementation in management and organization journals. Accountability in Research, 30(3), 150-175. doi:10.1080/08989621.2021.1981870
Hensel, P. G. (2023b). How often are replication attempts questioned? Accountability in Research, null-null. doi:10.1080/08989621.2023.2198126
Ioannidis, J. P. A. (2012). Why Science Is Not Necessarily Self-Correcting. Perspectives on Psychological Science, 7(6), 645-654. doi:10.1177/1745691612464056
Munafo, M. R., Nosek, B. A., Bishop, D. V. M., Button, K. S., Chambers, C. D., du Sert, N. P., . . . Ioannidis, J. P. A. (2017). A manifesto for reproducible science. Nature Human Behaviour, 1(1), 9. doi:10.1038/s41562-016-0021
Nosek, B. A., Alter, G., Banks, G. C., Borsboom, D., Bowman, S. D., Breckler, S. J., . . . Yarkoni, T. (2015). Promoting an open research culture. Science, 348(6242), 1422-1425. doi:10.1126/science.aab2374
Nosek, B. A., & Lakens, D. (2014). Registered Reports A Method to Increase the Credibility of Published Results. Social Psychology, 45(3), 137-141. doi:10.1027/1864-9335/a000192
Nosek, B. A., Spies, J. R., & Motyl, M. (2012). Scientific Utopia:II. Restructuring Incentives and Practices to Promote Truth Over Publishability. Perspectives on Psychological Science, 7(6), 615-631. doi:10.1177/1745691612459058
Shadish, W. R., Campbell, D. T., & Cook, T. D. (2001). Experimental and quasi-experimental designs for generalized causal inference. Boston: Houghton Mifflin.
Silberzahn, R., Uhlmann, E. L., Martin, D. P., Anselmi, P., Aust, F., Awtrey, E., . . . Nosek, B. A. (2018). Many Analysts, One Data Set: Making Transparent How Variations in Analytic Choices Affect Results. Advances in Methods and Practices in Psychological Science, 1(3), 337-356. doi:10.1177/2515245917747646
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant. Psychological Science, 22(11), 1359-1366. doi:10.1177/0956797611417632
Additional information
Additional information (registration calendar, class conductors, localization and schedules of classes), might be available in the USOSweb system: