Concept study / Tests & research methodsConceptual illustration. Shapes, colors, and motion are not measured data.
A screenshot of a plot is an illustration of a result, not the result’s full scientific record. For this project, computational reproducibility means another researcher can use the supplied inputs and procedure to obtain the stated output within an appropriate numerical tolerance. Replication means a new implementation, dataset or experiment tests the same claim. Both matter, and neither automatically establishes a universal theory. The practical goal is to lower the cost of checking a claim: make assumptions visible, provide a minimal runnable example, and preserve enough context that a future correction can be traced to the affected outputs.
Ship a complete analysis package
Include immutable raw data or a documented retrieval method, a data dictionary, preprocessing code, model code, dependency versions, random seeds and instructions. Record the input checksum and expected outputs. If data cannot be shared, explain the restriction and provide a permitted synthetic example with clearly labeled differences. A random seed alone does not guarantee identical results across software and hardware. State whether reproduction should be bit-for-bit or within numerical tolerance. The proposed research package should regenerate tables and figures directly so copied values cannot silently drift away from the underlying calculation.
Verify the pipeline on known answers
Test numerical routines against analytically known or independently computed benchmarks relevant to their risk. A box-counting implementation should recover sensible behavior on simple sets; a regression should recover injected parameters within expected uncertainty; an energy integration should handle a known waveform. These checks verify computation, not a speculative theory. Include failure examples where assumptions are violated so users can see limitations. Keep the smallest useful benchmark fast enough that contributors can run it before using the more expensive analysis on observational or experimental datasets.
Invite independent disagreement
Provide a clear route for an external researcher to submit a counterexample, corrected derivation or alternate implementation. Ask for the exact version, input and steps that produce the disagreement. Preserve the original output while publishing a correction and an impact note. For a claimed new effect, prioritize an independent setup with different likely artifacts rather than simply repeating the same apparatus many times. The project’s success should include becoming easier to falsify and repair. An analysis that others can inspect honestly can remain useful even when its initial interpretation does not survive.
The mathematical lens
A reproducibility dependency statement. All four inputs can affect the result and should be identified in the export.
Turn it into a question.
- Question
- Can an independent researcher regenerate the key claim?
- Method
- Run the package in a clean environment, then compare an independent implementation or dataset.
- What would count against it?
- Unexplained disagreement or missing inputs prevents a reproducible claim until resolved.
A useful research site lets readers check the work, not only read it.
Follow the original work.
These sources support the mechanisms and limits discussed here. They do not endorse Hill’s framework. Notes distinguish research proposals, reviews, and observations.
Continue the investigation
Edition 02 · 18 October 2026 · AI-assisted synthesis. See editorial method.