04 - Preregistration and Registered Reports

What they are and how to

Stefano Coretta

University of Edinburgh

Research process cycle

Questionable research practices (QRPs)

Preregistration

Preregistration is the practice of publicly specifying a study’s research questions/hypotheses, methods, and analysis plan in advance to control for analytic adequacy and increase transparency in research.

• You write a document with the plan.

• You upload it to a preregistration service (like OSF, aspredicted.org, …)

• It is time-stamped. You proceed with the study. No formal peer-review required.

Traditional publishing

Registered Reports

How to write an RR

Stage 1 manuscript

• Write introduction, background and methods.

• Target specific and clear RQs (and RHs) and assess feasibility (within constraints).

• Pilot studies or data simulations.

• Prepare a research compendium and ensure reproducibility.

In Principle Acceptance

Carry out the study according to the Stage 1 registered protocol.

Stage 2 manuscript

• Write results (of registered analyses + optional exploratory non-registered analyses), discussion and conclusion.

Resources

• See COS resources: https://www.cos.io/initiatives/registered-reports

• Chambers et al. (2015), Chambers and Tzavella (2021), Hardwicke and Ioannidis (2018), Karhulahti (2022), Karhulahti et al. (2023), Lakens et al. (2024).

Proportion of negative results

From Scheel et al. (2021).

Not killing the vibe

From Soderberg et al. (2021).

Guidance for PhD students

• PhD students can benefit from RRs.

• First year is critical: conceptualisation and writing of RR Stage 1.

• Time spent on preparing Stage 1 seems longer but time we should spend anyway. Overall time (from inception to publication) similar.

• Stage 2 review is very quick. Most issues with traditional review are about things that should have been done, but it’s too late.

Venues

• Subject specific journals that offer Registered Reports.

• PCI RR: https://rr.peercommunityin.org.

Make your next study a Registered Report!

References

Chambers, Christopher D., Zoltan Dienes, Robert D. McIntosh, Pia Rotshtein, and Klaus Willmes. 2015. “Registered Reports: Realigning Incentives in Scientific Publishing.” Cortex 66: A1A2. https://doi.org/10.1016/j.cortex.2015.03.022.

Chambers, Christopher D., and Loukia Tzavella. 2021. “The Past, Present and Future of Registered Reports.” Nature Human Behaviour 6 (1): 29–42. https://doi.org/10.1038/s41562-021-01193-7.

Hardwicke, Tom E., and John P. A. Ioannidis. 2018. “Mapping the Universe of Registered Reports.” Nature Human Behaviour 2 (11): 793–96. https://doi.org/10.1038/s41562-018-0444-y.

Karhulahti, Veli-Matti. 2022. “Registered Reports for Qualitative Research.” Nature Human Behaviour 6 (1): 45. https://doi.org/10.1038/s41562-021-01265-8.

Karhulahti, Veli-Matti, Peter Branney, Miia Siutila, and Moin Syed. 2023. “A Primer for Choosing, Designing and Evaluating Registered Reports for Qualitative Methods.” Open Research Europe 3: 22. https://doi.org/10.12688/openreseurope.15532.2.

Lakens, Daniël, Cristian Mesquida, Sajedeh Rasti, and Massimiliano Ditroilo. 2024. “The Benefits of Preregistration and Registered Reports.” Evidence-Based Toxicology 2 (1). https://doi.org/10.1080/2833373x.2024.2376046.

Scheel, Anne M., Mitchell R. M. J. Schijen, and Daniël Lakens. 2021. “An Excess of Positive Results: Comparing the Standard Psychology Literature with Registered Reports.” Advances in Methods and Practices in Psychological Science 4 (2): 25152459211007467. https://doi.org/10.1177/25152459211007467.

Soderberg, Courtney K., Timothy M. Errington, Sarah R. Schiavone, et al. 2021. “Initial Evidence of Research Quality of Registered Reports Compared with the Standard Publishing Model.” Nature Human Behaviour 5 (8): 990–97. https://doi.org/10.1038/s41562-021-01142-4.