Registered Reports as a model for research reform

Theory and practice

Stefano Coretta and Jessica Hampton

2026-05-21

Research crises

Traditional publishing

Registered Reports


See Chambers and Tzavella (2021) and https://www.cos.io/initiatives/registered-reports.

Tip 1 Precise RQs (and RHs)

Vague RQ

Do bilingual individuals have a cognitive advantage in executive functions?

Define all content words

  • What type of bilingualism?

  • Advantage in what sense?

  • Which executive functions? In which tasks? Within which constraints?

  • What mechanistic process model are we assuming?

Vague RQ

How are rivers represented in newspaper articles that talk about pollution?

Define all content words

  • What do we mean by representation?
    • Critical Discourse Analysis? Which framework?
  • Which newspapers? What time range?
  • What keywords?
  • What kind of pollution?

Tip 2 Detailed methodology

Quantitative

  • Specify operationalisation of all concepts.

  • Full analysis pipeline (not enough to say “we will use mixed-effects models”).

  • Sample size determination. Lakens (2022)

  • Causal inference (for variable selection). Rohrer (2018), Bailey et al. (2024)

  • Detailed description of statistical modelling, predictors, interactions, random/varying effects, prior specification, diagnostics.

  • Provide positive/negative controls, data validity checks.

  • Working code using simulated or pilot data.

Qualitative

  • Specify the framework and linguistic strategies of interest (even if exploratory).
  • Full explanation of how data is collected.
  • Provide a detailed description of how the analyses will be conducted.
  • Describe assumptions about the chosen data set.
  • Anything else? Karhulahti (2022), Karhulahti et al. (2023)

Tip 3 Research compendium

Research Compendium

A research compendium is a repository containing all materials, code, notebooks, images, data, metadata, manuscripts, etc of a project. A compendium is structured in a way that makes the research process transparent and reproducible.

  • Ideally, use a single main folder.
    • Compendia can belong to a super-project and have sub-project compendia/folders. A single paper can also be associated to a specific compendium.
  • Organise files and folders inside according to type and context: separate data, code, images.
  • Separate raw and derived data.
  • Use as much automation as possible.
  • Document everything (for example with READMEs).
  • Add a LICENSE file (for example, https://creativecommons.org/chooser/).
  • Use version control (like https://git-scm.com).

Tip 4 Levels of bias control

From PCI RR guidelines for authors.

Venues

  • Registered Reports in Linguistics (RRLing): https://journals.ed.ac.uk/rrling/index

  • Journal of Phonetics.

  • Language and Speech.

  • Language and Cognition.

  • Bilingualism: Language and Cognition.

  • Language Learning.

  • Journal of Child Language.

  • Glossa Psycholinguistics.

  • Cognitive Linguistics

  • Biolinguistics.

Make your next study a Registered Report!

Things we can talk about

References

Bailey, Drew H., Alexander J. Jung, Adriene M. Beltz, et al. 2024. “Causal Inference on Human Behaviour.” Nature Human Behaviour 8 (8): 1448–59. https://doi.org/10.1038/s41562-024-01939-z.
Bezeau, Scott, and Roger Graves. 2001. “Statistical Power and Effect Sizes of Clinical Neuropsychology Research.” Journal of Clinical and Experimental Neuropsychology 23 (3): 399–406. https://doi.org/10.1076/jcen.23.3.399.1181.
Chambers, Christopher D., and Loukia Tzavella. 2021. “The Past, Present and Future of Registered Reports.” Nature Human Behaviour 6 (1): 29–42. https://doi.org/10.1038/s41562-021-01193-7.
Cohen, Jacob. 1962. “The Statistical Power of Abnormal-Social Psychological Research: A Review.” The Journal of Abnormal and Social Psychology 65 (3): 145–53. https://doi.org/10.1037/h0045186.
Devezer, Berna, Luis G Nardin, Bert Baumgaertner, and Erkan Ozge Buzbas. 2019. “Scientific Discovery in a Model-Centric Framework: Reproducibility, Innovation, and Epistemic Diversity.” PloS One 14 (5): e0216125. https://doi.org/10.1371/journal.pone.0216125.
Fanelli, Daniele. 2010. “Do Pressures to Publish Increase Scientists’ Bias? An Empirical Support from US States Data.” PLoS ONE 5 (4): e10271. https://doi.org/10.1371/journal.pone.0010271.
John, Leslie K., George Loewenstein, and Drazen Prelec. 2012. “Measuring the Prevalence of Questionable Research Practices with Incentives for Truth Telling.” Psychological Science 23 (5): 524–32. https://doi.org/10.1177/0956797611430953.
Karhulahti, Veli-Matti. 2022. “Registered Reports for Qualitative Research.” Nature Human Behaviour 6 (1): 4–5. https://doi.org/10.1038/s41562-021-01265-8.
Karhulahti, Veli-Matti, Peter Branney, Miia Siutila, and Moin Syed. 2023. “A Primer for Choosing, Designing and Evaluating Registered Reports for Qualitative Methods.” Open Research Europe 3: 22. https://doi.org/10.12688/openreseurope.15532.2.
Kerr, Norbert L. 1998. “HARKing: Hypothesizing After the Results Are Known.” Personality and Social Psychology Review 2 (3): 196–217. https://doi.org/10.1207/s15327957pspr0203_4.
Lakens, Daniël. 2022. “Sample Size Justification.” Collabra: Psychology 8 (1): 33267. https://doi.org/10.1525/collabra.33267.
Makel, Matthew C., Jonathan A. Plucker, and Boyd Hegarty. 2012. “Replications in Psychology Research: How Often Do They Really Occur?” Perspectives on Psychological Science 7 (6): 537–42. https://doi.org/10.1177/1745691612460688.
Rohrer, Julia M. 2018. “Thinking Clearly about Correlations and Causation: Graphical Causal Models for Observational Data.” Advances in Methods and Practices in Psychological Science 1 (1): 2742. https://doi.org/10.1177/2515245917745629.
Scheel, Anne M. 2022. “Why Most Psychological Research Findings Are Not Even Wrong.” Infant and Child Development 31 (1): e2295. https://doi.org/10.1002/icd.2295.
Scheel, Anne M., Mitchell R. M. J. Schijen, and Daniël Lakens. 2021. “An Excess of Positive Results: Comparing the Standard Psychology Literature with Registered Reports.” Advances in Methods and Practices in Psychological Science 4 (2): 25152459211007467. https://doi.org/10.1177/25152459211007467.
Sedlmeier, Peter, and Gerd Gigerenzer. 1992. “Do Studies of Statistical Power Have an Effect on the Power of Studies?” In Methodological Issues & Strategies in Clinical Research. American Psychological Association. https://doi.org/10.1037/10109-032.
Soderberg, Courtney K., Timothy M. Errington, Sarah R. Schiavone, et al. 2021. “Initial Evidence of Research Quality of Registered Reports Compared with the Standard Publishing Model.” Nature Human Behaviour 5 (8): 990–97. https://doi.org/10.1038/s41562-021-01142-4.
Wicherts, Jelte M., Denny Borsboom, Judith Kats, and Dylan Molenaar. 2006. “The Poor Availability of Psychological Research Data for Reanalysis.” American Psychologist 61 (7): 726. https://doi.org/10.1037/0003-066X.61.7.726.
Yarkoni, Tal. 2022. “The Generalizability Crisis.” Behavioral and Brain Sciences 45. https://doi.org/10.1017/s0140525x20001685.

Proportion of negative results

From Scheel et al. (2021).

Not killing the vibe

From Soderberg et al. (2021).

Guidance for students/supervisors

  • PhD students especially can benefit from RRs.

    • Year one to develop topic and submit Stage 1.

    • Programmatic RRs.

  • Master students: Stage 1 manuscript as dissertation.

    • Start very early (Semester 1).

    • Arrangements for post-IPA.

How to write an RR

Stage 1 manuscript

Write introduction, background and methods.

In Principle Acceptance

Carry out the study according to the Stage 1 registered protocol.

Stage 2 manuscript

Write results (of registered analyses + optional exploratory non-registered analyses), discussion and conclusion.

Publication

Your Stage 2 manuscript is published.

See Chambers and Tzavella (2021) and https://www.cos.io/initiatives/registered-reports.