Get Involved

As a codechecker

You are a researcher, developer, research software engineer, student, or data/software steward and want to help codechecking submissions?

Awesome! Please open an issue on the codecheckers repository to be added to the codecheckers team using this link.

As an author

You want to share the code underlying your research paper? Congratulations for the decision to practice Open Science and reproducible research!

There are different ways for you to get involved, for example (a) submit a manuscript to a journal conducting CODECHECKs, (b) suggest a CODECHECK as part of your cover letter (see this template for a letter to the editor), or (c) submit your workflow to the community review process right away if you have published a preprint. You can also combine (b) and (c) by sharing a draft or your work, or a preprint link, with a CODECHECK editor. For (c), your first step is to propose a new CODECHECK by opening an issue on the CODECHECK register - see the author guide for how to prepare your workflow. In the eventual publication, please be so kind to reference the CODECHECK certificate and possibly also the project in your manuscript using the citation provided at the end of this page.

Your work was codechecked? Good job! Please consider adding a CODE WORKS badge to your repository or your personal website when you list your works. The badge should link directly to the Zenodo record using the DOI. The following snippet should work in Markdown:

[![CODECHECK](https://codecheck.org.uk/img/codeworks-badge.svg)](https://doi.org/<DOI HERE>)

Propose a new CODECHECK

You know of a paper whose code and data should be codechecked - your own, one you are reviewing or editing, or one your institution supports? New CODECHECKs start with an issue on the CODECHECK register. From the issue templates, your situation likely best matches a new community CODECHECK, i.e., for papers or preprints to be checked by a volunteer codechecker in the community workflow.

Note: The CODECHECK initiative is independent and driven by volunteers. This means that we have limited resources and it can take a while for us to respond to your proposal. Please understand that we may prioritise works based on our own experienes, the state of the work, existing collaborations, and available resources. For example, a published preprint or a privately shared PDF that is under submission at a journa we know both go a long way for us to better understand the state of a piece of research. Reach out to the team if you have any concerns about how your request is handled.

If you are proposing somebody else’s paper, please talk to the authors first - a CODECHECK works best when the authors are on board and can answer the codechecker’s questions. Not sure which template fits, or no GitHub account? Get in touch and we will open the issue with you.

As a reviewer or editor

Take a look at the benefits of a CODECHECK in a review process.

If you are contributing to science as a reviewer of academic manuscripts or as a journal editor, you can initiate a CODECHECK for submitted articles if the author provides suitable information. The CODECHECK team would be excited to help you start a conversation with your (fellow) journal editors on how you to best achieve that, i.e., if you conduct a community CODECHECK (which you can propose on the register), or if you want to set up a proper process at the journal (see below).

Would you like to support community CODECHECKs as a volunteer CODECHECK editor, i.e., to assign and mentor codecheckers and to publish certificates? Take a look at the workflow for CODECHECK editors and get in touch.

As a journal, publisher, or conference

You are an editor or reviewer at a journal or conference and are interested to introduce a CODECHECK?

Take a look at the benefits of a CODECHECK in a review process.

Still interested? Great! There is no formal process or certification. However, we are very interested in helping you to get started. Every venue implements the CODECHECK principles a little differently. The section on implementing CODECHECK describes the dimensions along which implementations differ, and the existing workflows show how other journals, conferences, and institutions have answered them. Before you start, it helps to think about the following questions:

Who conducts the checks - reviewers, editorial staff, a dedicated reproducibility committee, or volunteer codecheckers from the community?
When in the editorial process does the check happen - before peer review, in parallel to it, or after acceptance?
Which submissions are checked - all of them, those whose authors opt in, or a selection?
How open is the check - are the certificate and the codechecker’s name published, and can codecheckers and authors talk directly?
How important is the outcome - is a successful CODECHECK a requirement for publication, or an additional signal for editors and readers?

You don’t need to have all the answers yet! If you are interested in CODECHECKs at your venue, please get in touch with the CODECHECK team. We are happy to talk through these questions with you, share our experiences, and help you plan your first checks. The example process figure on the benefits page shows one way a CODECHECK process can fit into peer review, and the community workflow describes the tasks of authors, codecheckers, and CODECHECK editors in one concrete implementation.

Once your process is in place, let your authors and reviewers know about it. For example, you can announce publicly that you conduct workflow reviews adhering to the CODECHECK principles, e.g., in a blog post or editorial, and link to the principles and your process in your submission guidelines. For every completed check, please consider adding a badge to the publication to highlight the fact that a CODECHECK was conducted; the badge should link directly to the CODECHECK certificate. After your first CODECHECK, we would be happy to list your journal or conference on our partners page.

The badge graphic, the CODECHECK logo, and all documentation on this website are available under open licenses, see this README.

As a steward or service provider

You are a data steward, a research software steward, or any other non-research staff that helps researchers to create more FAIR and reproducible research contributions? Thank you for your service! We have several organisations (see this page with institutional CODECHECK workflows) that have implemented CODECHECK workflows as a service offering for their researchers and would be happy to share their experiences with you, and learn how we can help to get you started, or how we can help you to improve your existing code checking workflows. Once your service is running, each check is registered with the institutional CODECHECK issue template.

As a developer or science communicator

CODECHECK is a community effort, and your help is welcome across all tasks that the team faces. If you want to write code to assist codecheckers to conduct more effective reviews or to streamline the CODECHECK review processes, or if you want to help presenting and sharing CODECHECK’s vision and educate others on code executability checks - then please get in touch with the CODECHECK team (see contact details below).

As an event organiser

You would like to (co-)organise a CODECHECK event, such as a workshop or a codechecking exercise, to bring together codecheckers, authors, and editors? You want to introduce reproducibility checking into your community, maybe expand an existing event or series, such as a conference? We got you covered! Please check out our event recipe for a detailed guide on how to run a CODECHECK event.

As a curious person or someone with a question - get in touch!

Contact the PIs and senior codecheckers at team@cdchck.science. Personal emails for the PIs are in the page footer.

To cite CODECHECK in scientific publications, please use the following citation/reference:

Eglen, S., & Nüst, D. (2019). CODECHECK: An open-science initiative to facilitate the sharing of computer programs and results presented in scientific publications. Septentrio Conference Series, (1). https://doi.org/10.7557/5.4910

To get or give a quick overview of the project, feel free to use or extend the existing slide decks.