The raw data. Every file pinned by SHA-256.
Every run behind the studio’s research, with its receipts and outputs, and a manifest that pins each file by hash. It’s public because the research is open by default. The few files with a named reason to be private sit in a private companion, and no private file’s hash is ever published.
Clone
git clone https://github.com/mcp-tool-shop-org/rnd-data
Find a run
ls runs/ # one folder per run: manifest.json, files/
Check a file
sha256sum runs/<id>/files/outcome.json
What’s here
Manifests and small files in git; larger files once, by content.
A manifest per run
runs/<id>/manifest.json (schema rnd-data-run/v1) lists each file’s path, SHA-256, size, tier, licence and visibility, and every copy.
Small files in git
A run’s public files of 256 KB or less sit under runs/<id>/files/, stored byte-exact so every hash holds.
Larger files once
Files over 256 KB are stored once, by content, at blobs/<sha256[:2]>/<sha256>, so a build several runs used is kept once. They go public as a Hugging Face dataset at the next phase.
A run id is its folder
The source folder, with / written as --. A run is the nearest folder holding an outcome.json, so each retry is its own run; other files are grouped into a collection.
Read-only package under src/
The repository holds data, plus its MIT read-only Python package under src/ (rnd-data: ls, show, where, schema). rnd writes it (rnd raw ingest) and checks it (rnd raw verify). The .sh and .py files under runs/ are receipts of runs.
No machine paths
store.json names the stores that hold bytes outside the repo. Each machine finds their roots through a gitignored file or an environment variable.
Check a file without rnd
One file, by hand
sha256sum runs//files/
grep -A1 '"path": ""' runs//manifest.json
# the two hashes must match Every git copy (with jq)
jq -r '.copies[] | select(.store == "rnd-data")
| "\(.sha256) \(.location)"' runs/*/manifest.json \
| sha256sum -c --quietWith rnd
RND_DATA="$PWD" rnd raw verify --public-only
bash verify.sh # the same, as CI runs itPrivate files
Private only for a named reason. No private file’s hash is published.
| Why it’s private | What its run’s manifest says |
|---|---|
| Sealed or held-out keys, blind files, item text, unapproved plans, private projects | Nothing by name: it’s left out, and counted by reason in private_files |
| Only identity (the studio’s identity scan flagged it) | Its path and size, with "visibility": "private", "private_why": "identity" and "sha256": null |
| Anything else | It’s public, with its hash and its copies |
Why no private hash is published
A hash can give a short file away.
Hashes can be guessed
A short label file has few possible contents. Publishing its SHA-256 would let anyone confirm a guess, so a private file’s hash never appears here.
The bytes live elsewhere
A private file’s copy is in rnd-data-private, the private companion, or in the studio’s blob store. Nothing from either enters this repository.
Checked on every pull request
CI fails on a private entry with a hash, the private tag, a private-looking file name, a machine path in a manifest, or a file under runs/ that no manifest names.
Who owns what
| Who | What |
|---|---|
| R&D | The layout and the tooling: rnd writes every manifest and checks every copy |
| The Publisher | Its receipts, and a review of every ingest of them |
| The Director | The rulings on what is public, and the go before anything is deleted from the studio rig |