A process for independent reproduction of computations underlying biomedical research

CODE CHECK tackles one of the main challenges of computational biomedical research by supporting codecheckers with a workflow, guidelines and tools to evaluate computer programs underlying scientific papers. The independent time-stamped runs conducted by codecheckers will award a “certificate of reproducible computation” and increase availability, discovery and reproducibility of crucial artefacts for computational sciences. See the project page for a full description of problems, solutions, and goals.

The CODE CHECK principle: Codecheckers are human. They are supported by automation and reproducibility infrastructure.

How does a check work?

  1. A manuscript with computations passes peer-review and is assigned a codechecker.
  2. The codechecker must understand the instructions and executes the workflow using the CODE CHECK infrastructure. They contact the manuscript author if needed and collaborate based on the CODE CHECK infrastructure.
  3. A CODE CHECK evaluates if the code and data produce a core output (figure, dataset) - it is not a result check or judgement of scientific merit.
  4. The codechecker makes the final call on the result within a week, publishes the checked code and data together with the environment in a public repository (with DOI), and puts their name on it. A CODE CHECK badge badge on the published article points to the repository with the check files.

To stay in touch with the project follow the project team members on Twitter: