Community workflow
The CODECHECK community workflow described here is the steps that codecheckers follow when conducting independent CODECHECKs, i.e., without an active collaboration with a journal or conference. The community workflow uses free to use infrastructure, in particular the GitHub organisation codecheckers and Zenodo for depositing codecheck certificates. The community workflow is a concrete implementation of the abstract CODECHECK process and follows the CODECHECK principles, although it may fall short of Rule 3, because the CODECHECKs are not yet properly registered as peer reviews in open research information databases.
The community workflow documentation is split into three parts:
- CODECHECK community workflow for authors
- CODECHECK community workflow for codecheckers
This perspective has variants: a short version, an extended version, and a version for in-person workshop with CHECK-NL - CODECHECK community workflow for CODECHECK editors
This is the role of the CODECHECK editor; if you are a journal editor or publisher interested in CODECHECKs at your venue, see how to get involved as a journal, publisher, or conference.
Support
Every CODECHECK is unique, just as the associated research article. The codechecker can partly rely on the examples below and the full breadth of CODECHECKs for good practices and approaches for codechecking.
If you have any questions or concerns, or get stuck, please reach out to fellow codecheckers in the public CODECHECK discussion forum or contact the CODECHECK team (see footer) if you need to consult on sensitive information.
Examples
See the CODECHECK register for a full list of codechecks, including direct links to the reports and register issues. You can check there for more examples from different communities. The good practices and common pitfalls from the examples below are summarised in the author guide.
- Life Sciences 🧬🌿🧠
-
ShinyLearner: A containerized benchmarking tool for machine-learning classification of tabular data
CODECHECK certificate 2020-001: https://doi.org/10.5281/zenodo.3674056
Look at this for: recreating figures from provided result files when the full computation takes days; dependencies and manifest the codechecker had to add.
-
CODECHECK certificate 2020-010: https://doi.org/10.5281/zenodo.3865491
Look at this for: a large simulation run on an HPC cluster, and how to document differences between original and reproduced numbers.
-
Driftage: a multi-agent system for concept drift detection and an application on electromyography (Uses MyBinder)
CODECHECK certificate 2020-024: https://doi.org/10.5281/zenodo.4964880
Look at this for: a MyBinder link that made the check take a few minutes.
-
- GIScience/Geography/Geoinformatics 🌏🌎🌍
-
Progress in the R ecosystem for representing and handling spatial data
CODECHECK certificate 2020-017: https://doi.org/10.5281/zenodo.4003848
Look at this for: a README listing data, script, and outputs; separately saved figures; captured console output for comparison.
-
Geographically Weighted Regressions in educational planning (R and QGIS)
CODECHECK certificate 2022-001: https://doi.org/10.5281/zenodo.6040066
Look at this for: a video documenting manual GIS steps; data available on request; Windows-specific paths.
-
Is it safe to be attractive? Disentangling the influence of streetscape features on the perceived safety and attractiveness of city streets (web app, Jupyter notebook)
CODECHECK certificate 2023-008: https://doi.org/10.17605/OSF.IO/AQGXR
Look at this for: a web application started with one
docker-composecommand; anonymised data shared after ethics approval. -
Urban Sound Mapping for Wayfinding - A theoretical Approach and an empirical Study (R and QGIS)
CODECHECK certificate 2023-001: https://doi.org/10.17605/OSF.IO/C7VX3
Look at this for: why to map data files to figures and tables, and to script aggregation steps instead of doing them in a spreadsheet.
-
Road Network Mapping from Multispectral Satellite Imagery: Leveraging Deep Learning and Spectral Bands (Python with Conda, data subset)
CODECHECK certificate 2024-013: https://doi.org/10.17605/OSF.IO/TXGZV
Look at this for: a small preprocessed data subset in place of a 100 GB dataset; CPU-only reproduction of a GPU workflow.
-
The Impact of Traffic Lights on Modal Split and Route Choice: A use-case in Vienna (Python, with requirements.txt by codechecker)
CODECHECK certificate 2024-012: https://doi.org/10.17605/OSF.IO/W42AD
Look at this for: published intermediate results that allow checking later steps without re-running a 16-hour step.
-
Developing a city-specific walkability index through a participatory approach (mixed methods; QGIS and Python)
CODECHECK certificate 2024-009: https://doi.org/10.17605/OSF.IO/CSB7R
Look at this for: how manual preprocessing limits a check to part of the workflow.
-
Knowledge-Based Identification of Urban Green Spaces: Allotments (R, proprietary GIS)
CODECHECK certificate 2024-008: https://doi.org/10.17605/OSF.IO/3NBJW
Look at this for: a sample dataset and an ArcGIS project package when input data is not freely available; undocumented memory requirements.
-
- Archaeology ⛏️🗿
-
CODECHECK certificate 2020-025: https://doi.org/10.5281/zenodo.4279275
Look at this for: a README that explains what each script needs and creates.
-
- Your scientific discipline here…
CODECHECK