Reproducibility & replication

No Action (Quant.)

No steps are taken to make the research reproducible or to support replicability. Data, code, and procedures are not shared, and others cannot verify, rerun, or independently test the findings.

Moving from No Action to Emerging in Reproducibility & Replicability (Quant.)

1. To progress from No Action to Emerging, start by organising your materials and documenting your workflow so that others could, in principle, re-run your analysis if requested.
2. Organise your data and code. Store your datasets, analysis scripts, and outputs in one structured location. Use a clear folder system that separates raw data, cleaned data, and analysis files.
3. Document your workflow. Write down the steps you took to clean, prepare, and analyse the data. Even brief comments within your code or a simple summary document provides an initial record of your process.
4. Record software and tools. List the software, packages, and version numbers used in your analysis (for example, sessionInfo() in R or pip freeze in Python). This helps others understand the technical environment behind your results.
5. Preserve a clean version of the dataset. Save an unaltered copy of the dataset you used for analysis. Ensure variables are clearly labelled and documentation exists for what each variable represents.
6. Share informally when requested. Be prepared to provide your dataset and code on request (for example, to reviewers or colleagues), even if you are not yet sharing publicly through a repository.
7. Begin exploring preregistration. Familiarise yourself with preregistration platforms such as the Open Science Framework (OSF) Registries. You do not need to preregister yet, but understanding the process prepares you for later stages.

Emerging (Quant.)

Reproducibility is acknowledged and partially supported, for example by sharing selected datasets or code on request. However, documentation is limited and key components are still missing, meaning that others would struggle to fully reproduce the analysis.

Moving from Emerging to Evolving in Reproducibility & Replicability (Quant.)

1. To progress from Emerging to Evolving, focus on moving from informal or partial sharing toward structured, publicly accessible materials accompanied by basic documentation.
2. Share data and code publicly. Deposit your cleaned dataset and analysis scripts in a trusted repository such as the Open Science Framework (OSF), Zenodo, or GitHub, making them openly accessible wherever ethical and legal constraints permit.
3. Add basic documentation. Include a README file that summarises the purpose of the dataset, defines each variable, explains file structure, and provides simple instructions for running the analysis scripts.
4. Improve code clarity. Add explanatory comments throughout your code, organise it into logical sections (for example, data preparation, analysis, and visualisation), and use consistent naming conventions to improve readability.
5. Begin preregistration for confirmatory research. For hypothesis-driven studies, preregister your design and planned analyses using platforms such as OSF Registries. This helps distinguish planned from exploratory work.
6. Report deviations transparently. If changes are made after preregistration, explain what changed, why, and how it may affect interpretation of the results.
7. Share negative and exploratory results. Report all findings -- including null or exploratory results -- either in the manuscript or in supplementary materials to reduce publication bias and provide a more accurate record of the research.

Evolving (Quant.)

Reproducibility is actively supported by sharing full datasets, analysis code, and clear methodological detail in public repositories. Others can rerun the analysis with relative ease. Replication is recognised as valuable, and attempts to replicate findings are occasionally undertaken or encouraged.

Moving from Evolving to Sustained in Reproducibility & Replicability (Quant.

1. To progress from Evolving to Sustained, the goal is not only to share reproducible materials, but to make reproducibility a routine, fully documented, and citable part of your research workflow.
2. Provide comprehensive documentation (Beyond a Basic README). Replace simple READMEs with full documentation packets that include a detailed project overview, variable definitions, file structure, analytic rationale, dependencies, and instructions for reproducing the workflow end-to-end.
3. Strengthen code quality and transparency. Ensure scripts are modular, well-commented, and version-controlled. Use reproducible practices such as workflow automation (for example, make, targets, renv, or virtualenv) so analyses can be rerun by others with minimal friction.
4. Make preregistration routine. Instead of preregistering occasionally, preregister all confirmatory studies as a default practice. Use structured preregistration templates that clearly distinguish confirmatory from exploratory analyses.
5. Document deviations as a changelog. Maintain a publicly accessible record of changes, including deviations from preregistered plans and justifications. This creates a clear audit trail and supports interpretive accountability.
6. Publish all results transparently. Make full results available, including robustness tests, null and exploratory findings, and alternative model specifications. Provide supplementary files when needed to allow others to assess analytic decisions.
7. Archive for longevity and citation. Assign DOIs to datasets, analysis scripts, and workflows using repositories such as Zenodo or OSF so they are permanently citable and discoverable.

Sustained (Quant.)

Reproducibility is systematically embedded throughout the research workflow. Data, code, and analytic decisions are fully documented and openly archived, enabling others to rerun the study without additional clarification. Replication is actively supported and encouraged, either through direct replication efforts or by designing studies to be easily replicable. The researcher also promotes reproducibility practices within their research community, contributing to a broader culture of transparency and methodological rigor.

Guidance for Sustained Level in Reproducibility & Replicability

Congratulations on reaching this level of practice. You are operating beyond good practice and contributing as a field leader in open and ethically grounded research. At the sustained level, reproducibility is not just practiced -- it is actively championed and built into the broader research ecosystem. The emphasis shifts from individual compliance to field-wide leadership and capacity-building. To maintain and further improve at this level, consider the following:

• Publish as registered reports where possible. Submit studies as Registered Reports so that peer review occurs before data collection. This improves methodological rigour, reduces publication bias, and signals a commitment to transparency from the outset.
• Lead and support replication efforts. Organise or participate in direct and conceptual replications, whether of your own studies or influential findings in your field. This helps strengthen evidence bases and contributes to cumulative science.
• Develop and share open workflows. Create reproducible research pipelines using tools such as R Markdown, Quarto, Jupyter notebooks, or workflow managers like Snakemake. Sharing these pipelines enables others to rerun and adapt your analyses with minimal friction.
• Shape methodological standards. Contribute to working groups, consortia, or guideline committees that define best practices in reproducibility. This positions you as a leader in setting standards, not merely following them.
• Mentor and train others. Embed reproducibility in supervision and lab culture. Provide workshops, tutorials, or resources so that trainees and collaborators develop strong reproducible research habits early in their careers.
• Contribute to meta-research. Study and publish on reproducibility, transparency, and methodological reform within your field. Help generate the evidence base for how reproducibility improves scientific quality and impact.
• Shape institutional policy and infrastructure. Advocate for reproducibility to be reflected in promotion criteria, research training requirements, or internal funding guidelines. Help embed these expectations beyond the level of individual projects.
• Foster a collaborative culture. Promote collective ownership of reproducibility through shared templates, lab code libraries, standardised data documentation, and peer review of analysis workflows within your group or network.