Reproducibility infrastructure for AI

Every experiment should be reproducible.

Reprova automatically captures the code, environment, models, prompts, parameters, inputs and outputs behind every AI experiment, so you can reproduce exactly what happened later.

Join the waitlist See how it works

recording

exp_01824

$reprova run python eval.py

Starting experiment...

Code commit91fc8d2

Environmentcaptured

Modelclaude-sonnet

Promptsaved

Parametersrecorded

Inputsversioned

Outputsstored

Experiment recorded.

reprova/run/exp_01824

One command. The full execution context, written down.

01The problem

Six weeks later, can you reproduce this result?

Modern AI experiments depend on state that lives in a dozen places at once. The code is in git. The prompt is in a file somebody renamed twice. The parameters were typed into a shell that has since scrolled away. The model version is whatever the provider was serving that afternoon.

Any one of these changes the result. Most of them change quietly. By the time a number matters enough to defend, the context that produced it is gone.

artifact

model-v3

parameter

temperature=0.4

source

commit 81fd23

prompt

prompt_final_FINAL.txt

inputs

dataset_v7.csv

runtime

Python 3.12

parameter

seed=42

hardware

CUDA 13

Eight fragments, and the run is still not reproducible. You need all of them, at the same version, at the same moment, with the outputs they produced. That record either exists or it does not.

Reprova is a flight recorder for AI experiments.

An aircraft does not decide what to record after the incident. The recorder runs on every flight, whether or not anyone expects to need it. Research deserves the same default.

02How it works

Capture the run, then reconstruct it.

Reprova sits between your command and your experiment. It records the execution context on the way through, then uses that record to rebuild the run on demand.

Prefix the command you already run

Reprova starts a run around your process. It resolves the git state, snapshots the environment, and intercepts the model calls the process makes. There is no SDK to adopt and no decorator to add to every function. If the script runs today, it records today.

reprova run python eval.py

Everything is written down as it happens

Reprova writes an immutable record of the run: source revision and working tree diff, resolved dependency graph, driver and hardware versions, every prompt and completion, sampling parameters, seeds, dataset checksums and output artifacts. Records are content addressed, so identical inputs resolve to identical references across machines.

exp_01824 → 1,412 recorded fields

Reconstruct the run later

Reprova rebuilds the environment, restores the exact inputs, and replays the experiment against the recorded model and parameters. Where the new run diverges, you get a field by field diff instead of a hypothesis. You find out what changed, not just that something did.

reprova reconstruct exp_01824

What gets captured

code.commit: Source revision, uncommitted working tree diff, and submodule state at the moment the run started.
env.lock: Interpreter version, fully resolved dependency graph, system libraries, accelerator and driver versions.
model.ref: Provider, model identifier, and the version string the API actually returned, not the alias you asked for.
prompt.body: Every prompt, system message and tool definition sent during the run, stored verbatim.
params.set: Temperature, top_p, token limits, stop sequences, random seeds, and retry behavior.
data.inputs: Content addressed checksums for every dataset, fixture and file the run reads.
data.outputs: Completions, artifacts, metrics and logs, linked back to the inputs that produced them.
run.lineage: Parent runs, forks and retries, so a result can be traced through every experiment it descends from.

03Why Reprova

Reprova is not an experiment tracker.

Trackers ask you to log what you predict will matter. Reprova records what actually happened. The difference shows up months later, when the thing you needed is the thing you did not think to write down.

Capture is automatic, not remembered

A tracker is only as complete as the last person who remembered to call log(). Reprova records the execution context by default, including the parts nobody thought to instrument. Completeness is the product.

The record is the deliverable, not the chart

Dashboards summarize what happened. Reprova reconstructs it. Every run resolves back to an executable state you can rerun, diff, and hand to a colleague, a reviewer or an auditor.

Provenance is a graph, not a row

Results descend from other results. Reprova keeps the lineage intact, so a number in a report can be traced back through the runs, datasets and prompts that produced it.

04For teams

Reproducibility is an organizational problem first.

One researcher can hold a month of context in their head. A team cannot. Reprova gives a group of people a shared, checkable account of what was run.

Review: A reviewer can rerun the claim instead of trusting a screenshot in a document. Disagreements resolve against the record rather than against memory.
Handoff: An experiment outlives the person who ran it. New team members inherit runnable history instead of an oral tradition and a folder of scripts.
Regression: When a pipeline changes behavior without an obvious cause, the diff between two records tells you which field moved and when it moved.
Audit: Model and safety evaluations ship with provenance attached, so results hold up to scrutiny from people who were not in the room.

Early access

Write down the experiment while it is still running.

Reprova is in private development. We are working with a small number of research groups and applied AI teams before a public release. If reproducibility is a problem you actually have, tell us and we will be in touch.