A process for independent reproduction of computations underlying biomedical research
CODE CHECK tackles one of the main challenges of computational biomedical research by supporting codecheckers with a workflow, guidelines and tools to evaluate computer programs underlying scientific papers. The independent time-stamped runs conducted by codecheckers will award a “certificate of reproducible computation” and increase availability, discovery and reproducibility of crucial artefacts for computational sciences. See the project page for a full description of problems, solutions, and goals.
The CODE CHECK principle: Codecheckers are human. They are supported by automation and reproducibility infrastructure.
How does a check work?
- A manuscript with computations passes peer-review and is assigned a codechecker.
- The codechecker must understand the instructions and executes the workflow using the CODE CHECK infrastructure. They contact the manuscript author if needed and collaborate based on the CODE CHECK infrastructure.
- A CODE CHECK evaluates if the code and data produce a core output (figure, dataset) - it is not a result check or judgement of scientific merit.
- The codechecker makes the final call on the result within a week, publishes the checked code and data together with the environment in a public repository (with DOI), and puts their name on it. A badge on the published article points to the repository with the check files.
To stay in touch with the project follow the project team members on Twitter: