Registered Reports as a model for research reform

Theory and practice

Stefano Coretta and Jessica Hampton

2026-05-21

Research crises

Traditional publishing

Registered Reports

How to write an RR

Stage 1 manuscript

Write introduction, background and methods.

In Principle Acceptance

Carry out the study according to the Stage 1 registered protocol.

Stage 2 manuscript

Write results (of registered analyses + optional exploratory non-registered analyses), discussion and conclusion.

Publication

Your Stage 2 manuscript is published.

See Chambers and Tzavella (2021) and https://www.cos.io/initiatives/registered-reports.

Levels of bias control

Tip 1 Precise RQs (and RHs)

Vague RQ

Do bilingual individuals have a cognitive advantage in executive functions?

Define all content words

• What type of bilingualism?

• Advantage in what sense?

• Which executive functions? In which tasks? Within which constraints?

• What mechanistic process model are we assuming?

Vague RQ

How are rivers represented in newspaper articles that talk about pollution?

Define all content words

• What do we mean by representation?
  • Critical Discourse Analysis? Which framework?
• Which newspapers? What time range?
• What keywords?
• What kind of pollution?

Tip 2 Detailed methodology

Quantitative

• Specify operationalisation of all concepts.

• Full analysis pipeline (not enough to say “we will use mixed-effects models”).

• Sample size determination. Lakens (2022)

• Causal inference (for variable selection). Rohrer (2018), Bailey et al. (2024)

• Detailed description of statistical modelling, predictors, interactions, random/varying effects, prior specification, diagnostics.

• Provide positive/negative controls, data validity checks.

• Working code using simulated or pilot data.

Qualitative

• Specify the framework and linguistic strategies of interest (even if exploratory).
• Full explanation of how data is collected.
• Provide a detailed description of how the analyses will be conducted.
• Describe assumptions about the chosen data set.
• Anything else? Karhulahti (2022), Karhulahti et al. (2023)

Tip 3 Research compendium

Tip 4 Assess feasibility

Venues

• Registered Reports in Linguistics (RRLing): https://journals.ed.ac.uk/rrling/index

• Journal of Phonetics.

• Language and Speech.

• Bilingualism: Language and Cognition.

• Glossa Psycholinguistics.

Make your next study a Registered Report!

Things we can talk about

• Research crises/cycle/QRPs.

• RRLing and PCI RR.

• Reviewing RRs.

• RRs for PGT/PGR students.

• Language Learning Early Career Research Grant Program.

• And anything else you’d like!

References

Bailey, Drew H., Alexander J. Jung, Adriene M. Beltz, et al. 2024. “Causal Inference on Human Behaviour.” Nature Human Behaviour 8 (8): 1448–59. https://doi.org/10.1038/s41562-024-01939-z.

Bezeau, Scott, and Roger Graves. 2001. “Statistical Power and Effect Sizes of Clinical Neuropsychology Research.” Journal of Clinical and Experimental Neuropsychology 23 (3): 399–406. https://doi.org/10.1076/jcen.23.3.399.1181.

Chambers, Christopher D., and Loukia Tzavella. 2021. “The Past, Present and Future of Registered Reports.” Nature Human Behaviour 6 (1): 29–42. https://doi.org/10.1038/s41562-021-01193-7.

Cohen, Jacob. 1962. “The Statistical Power of Abnormal-Social Psychological Research: A Review.” The Journal of Abnormal and Social Psychology 65 (3): 145–53. https://doi.org/10.1037/h0045186.

Devezer, Berna, Luis G Nardin, Bert Baumgaertner, and Erkan Ozge Buzbas. 2019. “Scientific Discovery in a Model-Centric Framework: Reproducibility, Innovation, and Epistemic Diversity.” PloS One 14 (5): e0216125. https://doi.org/10.1371/journal.pone.0216125.

Fanelli, Daniele. 2010. “Do Pressures to Publish Increase Scientists’ Bias? An Empirical Support from US States Data.” PLoS ONE 5 (4): e10271. https://doi.org/10.1371/journal.pone.0010271.

John, Leslie K., George Loewenstein, and Drazen Prelec. 2012. “Measuring the Prevalence of Questionable Research Practices with Incentives for Truth Telling.” Psychological Science 23 (5): 524–32. https://doi.org/10.1177/0956797611430953.

Karhulahti, Veli-Matti. 2022. “Registered Reports for Qualitative Research.” Nature Human Behaviour 6 (1): 4–5. https://doi.org/10.1038/s41562-021-01265-8.

Karhulahti, Veli-Matti, Peter Branney, Miia Siutila, and Moin Syed. 2023. “A Primer for Choosing, Designing and Evaluating Registered Reports for Qualitative Methods.” Open Research Europe 3: 22. https://doi.org/10.12688/openreseurope.15532.2.

Kerr, Norbert L. 1998. “HARKing: Hypothesizing After the Results Are Known.” Personality and Social Psychology Review 2 (3): 196–217. https://doi.org/10.1207/s15327957pspr0203_4.

Lakens, Daniël. 2022. “Sample Size Justification.” Collabra: Psychology 8 (1): 33267. https://doi.org/10.1525/collabra.33267.

Makel, Matthew C., Jonathan A. Plucker, and Boyd Hegarty. 2012. “Replications in Psychology Research: How Often Do They Really Occur?” Perspectives on Psychological Science 7 (6): 537–42. https://doi.org/10.1177/1745691612460688.

Rohrer, Julia M. 2018. “Thinking Clearly about Correlations and Causation: Graphical Causal Models for Observational Data.” Advances in Methods and Practices in Psychological Science 1 (1): 2742. https://doi.org/10.1177/2515245917745629.

Scheel, Anne M., Mitchell R. M. J. Schijen, and Daniël Lakens. 2021. “An Excess of Positive Results: Comparing the Standard Psychology Literature with Registered Reports.” Advances in Methods and Practices in Psychological Science 4 (2): 25152459211007467. https://doi.org/10.1177/25152459211007467.

Sedlmeier, Peter, and Gerd Gigerenzer. 1992. “Do Studies of Statistical Power Have an Effect on the Power of Studies?” In Methodological Issues & Strategies in Clinical Research. American Psychological Association. https://doi.org/10.1037/10109-032.

Soderberg, Courtney K., Timothy M. Errington, Sarah R. Schiavone, et al. 2021. “Initial Evidence of Research Quality of Registered Reports Compared with the Standard Publishing Model.” Nature Human Behaviour 5 (8): 990–97. https://doi.org/10.1038/s41562-021-01142-4.

Wicherts, Jelte M., Denny Borsboom, Judith Kats, and Dylan Molenaar. 2006. “The Poor Availability of Psychological Research Data for Reanalysis.” American Psychologist 61 (7): 726. https://doi.org/10.1037/0003-066X.61.7.726.

Proportion of negative results

From Scheel et al. (2021).

Not killing the vibe

From Soderberg et al. (2021).

Guidance for students/supervisors

• PhD students especially can benefit from RRs.

  • Year one to develop topic and submit Stage 1.

  • Programmatic RRs.

• Master students: Stage 1 manuscript as dissertation.

  • Start very early (Semester 1).

  • Arrangements for post-IPA.