Open Research practices for linguistics

Stefano Coretta

University of Edinburgh

2026-03-18

The research crisis

  • Pashler and Wagenmakers (2012): “a crisis of confidence?”.

  • Yarkoni (2022): lack of meaningful connection between verbal generalisations and evidence.

  • Scheel et al. (2020): we are not even wrong.

Photo by Nik on Unsplash

Research reliability

From The Turing Way.

Research reliability

From The Turing Way.

Research reliability

Research soundness

Open Research

What is it?

Open Research is a movement that stresses the importance of a more honest and transparent research by promoting a series of research principles and by warning about common, although not necessarily intentional, questionable practices and misconceptions (Munafò et al. 2017; Crüwell et al. 2019; Casillas et al. 2025).

How to make your research open?

  • Share Research compendia.

  • Write Registered Reports.

  • Reflect on your researcher’s orientation.

Research compendium

A research compendium accompanies, enhances, or is a scientific publication providing data, code, and documentation for reproducing a scientific workflow.

—Research compendium

A research compendium is a collection of all digital parts of a research project including data, code, texts (protocols, reports, questionnaires, meta data). The collection is created in such a way that reproducing all results is straightforward.

—The Turing Way: Research compendia

Research compendium

Research Compendium

A research compendium is a repository containing all materials, code, notebooks, images, data, metadata, manuscripts, etc of a project. A compendium is structured in a way that makes the research process transparent and reproducible.

  • Ideally, use a single main folder.
  • Organise files and folders inside according to type and context: separate data, code, images.
  • Separate raw and derived data.
  • Use as much automation as possible.
  • Document everything (for example with READMEs).

Research compendium: bad example

Research compendium: good example

Sharing research compendia

Licensing

Pick a license

  • Creative Commons is a commonly chosen license: https://creativecommons.org/chooser/

  • Other licenses (for software): MIT License, GNU license.

  • Always include a LICENSE file in your compendium and be explicit which parts of the compendium fall under which license.

Registered Reports

Registered Reports

Registered Reports: positive results

From Scheel, Schijen, and Lakens (2021).

Registered Reports: not killing the vibe

From Soderberg et al. (2021).

Registered Reports: resources

Researcher’s orientation

Research is not done in a vacuum. Knowledge is contextual.

Researcher’s orientation

  • Reflexive understanding of one’s own individual aspects and how they shape one’s own approach to and practice of research.

  • Not limited to: identity, lived experiences, social positionality, philosophical stance, personal beliefs, methodological theory, and more.

Researcher’s orientation: example 1 (summary)

I identify as a neurodiverse male academic of Mediterranean ethnicity. I was born and raised in Northern Italy, but I have been living in the United Kingdom for a decade and I think of this country as my home. Most of my current research is about research methods and how research practices shape linguistic research. In this sense, I have an in-group point of view, being myself a researcher. My philosophical stance is a syncretic integration of non-dual monism, anti-realism, holism, cosmopsychism, spiritualism, subjective Bayesian epistemology, and ecological awareness. From the point of view of methodological theory and practice, I am an active advocate of Open Research practices for a more transparent, well-grounded and reflexive scholarship.

—Stefano Coretta (quantitative researcher). Full orientation statement: https://stefanocoretta.github.io/orientation.html

Researcher’s orientation: example 2

Building on the positionality articulated in my doctoral research, I continue to view the social world as inseparable from the environments—human and more-than-human—that shape it. During my PhD I recognised this worldview as resonant with posthumanist thought and situated my work within both poststructuralist and posthumanist traditions, understanding reality as plural, situated, and in constant flux. In line with this principle of flux, my focus has evolved. I now describe myself as an environmental sociolinguist informed by a more-than-human ontology, seeking to explore how language can help to reconcile the wounds of the contemporary polycrisis through processes of trauma acknowledgement and healing. My own trajectory also informs this stance: I grew up in northern Italy, and I have found a second home in Liverpool—a city whose struggle and resilience echo my own past. I write as a spiritual being attentive to connection and interconnectedness, committed to an applied linguistics that engages with the living world in all its entanglements.

—Jessica Hampton (qualitative and mixed methods researcher)

Researcher’s orientation: resources

  • Positionality

    • Darwin Holmes (2020), Goundar (2025) for qualitative research.

    • Jafar (2018), Lazard and McAvoy (2020) for quantitative research.

  • Philosophical stance

    • Tomlinson (2023), Castanelli (2024) for PhD/ECRs.

Summary

Open Research

  • Transparency and reliability of results (reproducible, replicable, robust, generalisable).

  • Share research compendia (with license).

  • Publish Registered Reports.

  • Think about your researcher’s orientation.

References

Casillas, Joseph V., Gabriela Constantin-Dureci, Iván Andreu Rascón, Jiawei Shao, Stephanie A. Rodríguez, Adrija Gadamsetty, Alexandria Minetti, et al. 2025. “Opening Open Science to All: Demystifying Reproducibility and Transparency Practices in Linguistic Research.” Linguistics. https://doi.org/doi:10.1515/ling-2023-0249.
Castanelli, Damian. 2024. “Developing Your Philosophical Stance as a PhD Student: A Case Study.” Focus on Health Professional Education: A Multi-Professional Journal 25 (2): 130–43. https://doi.org/10.11157/fohpe.v25i2.831.
Chambers, Christopher D., and Loukia Tzavella. 2021. “The Past, Present and Future of Registered Reports.” Nature Human Behaviour 6 (1): 29–42. https://doi.org/10.1038/s41562-021-01193-7.
Crüwell, Sophia, Johnny van Doorn, Alexander Etz, Matthew C. Makel, Hannah Moshontz, Jesse Niebaum, Amy Orben, Sam Parsons, and Michael Schulte-Mecklenbeck. 2019. “Seven Easy Steps to Open Science: An Annotated Reading List.” Zeitschrift Für Psychologie 227 (4): 237248. https://doi.org/10.1027/2151-2604/a000387.
Darwin Holmes, Andrew Gary. 2020. “Researcher Positionality: A Consideration of Its Influence and Place in Qualitative Research. A New Researcher Guide.” Shanlax International Journal of Education 8 (4): 110. https://doi.org/10.34293/education.v8i4.3232.
Devezer, Berna, Danielle J. Navarro, Joachim Vandekerckhove, and Erkan Ozge Buzbas. 2021. “The Case for Formal Methodology in Scientific Reform.” Royal Society Open Science 8 (3): rsos.200805, 200805. https://doi.org/10.1098/rsos.200805.
Goundar, Prashneel Ravisan. 2025. “Researcher Positionality: Ways to Include It in a Qualitative Research Design.” International Journal of Qualitative Methods 24 (March): 16094069251321251. https://doi.org/10.1177/16094069251321251.
Jafar, Anisa J. N. 2018. “What Is Positionality and Should It Be Expressed in Quantitative Studies?” Emergency Medicine Journal. https://doi.org/10.1136/emermed-2017-207158.
Karhulahti, Veli-Matti. 2022. “Registered Reports for Qualitative Research.” Nature Human Behaviour 6 (1): 45. https://doi.org/10.1038/s41562-021-01265-8.
Karhulahti, Veli-Matti, Peter Branney, Miia Siutila, and Moin Syed. 2023. “A Primer for Choosing, Designing and Evaluating Registered Reports for Qualitative Methods.” Open Research Europe 3: 22. https://doi.org/10.12688/openreseurope.15532.2.
Lazard, Lisa, and Jean McAvoy. 2020. “Doing Reflexivity in Psychological Research: What’s the Point? What’s the Practice?” Qualitative Research in Psychology 17 (2): 159–77. https://doi.org/10.1080/14780887.2017.1400144.
Munafò, Marcus R., Brian A. Nosek, Dorothy V. M. Bishop, Katherine S. Button, Christopher D. Chambers, Nathalie Percie Du Sert, Uri Simonsohn, Eric-Jan Wagenmakers, Jennifer J. Ware, and John P. A. Ioannidis. 2017. “A Manifesto for Reproducible Science.” Nature Human Behaviour 1 (1): 21. https://doi.org/10.1038/s41562-016-0021.
Pashler, Harold, and Eric-Jan Wagenmakers. 2012. “Editors’ Introduction to the Special Section on Replicability in Psychological Science: A Crisis of Confidence?” Perspectives on Psychological Science 7 (6): 528530. https://doi.org/10.1177/1745691612465253.
Scheel, Anne M., Mitchell R. M. J. Schijen, and Daniël Lakens. 2021. “An Excess of Positive Results: Comparing the Standard Psychology Literature With Registered Reports.” Advances in Methods and Practices in Psychological Science 4 (2): 25152459211007467. https://doi.org/10.1177/25152459211007467.
Scheel, Anne M., Leonid Tiokhin, Peder M. Isager, and Daniël Lakens. 2020. “Why Hypothesis Testers Should Spend Less Time Testing Hypotheses.” Perspectives on Psychological Science 16 (4): 744755. https://doi.org/10.1177/1745691620966795.
Soderberg, Courtney K., Timothy M. Errington, Sarah R. Schiavone, Julia Bottesini, Felix Singleton Thorn, Simine Vazire, Kevin M. Esterling, and Brian A. Nosek. 2021. “Initial Evidence of Research Quality of Registered Reports Compared with the Standard Publishing Model.” Nature Human Behaviour 5 (8): 990–97. https://doi.org/10.1038/s41562-021-01142-4.
Tomlinson, Yalda Natasha. 2023. “The Importance of Engaging with Ontology and Epistemology as an ECR.” https://www.bps.org.uk/psychologist/importance-engaging-ontology-and-epistemology-ecr.
Yarkoni, Tal. 2022. “The Generalizability Crisis.” Behavioral and Brain Sciences 45. https://doi.org/10.1017/s0140525x20001685.