Skip to content

Geometry

A geometry file is the export that aspire-si writes for one training run:

  • the student’s states projected to two dimensions;
  • each evaluator’s scores per step;
  • the eigen spectrum of the states;
  • the evaluators’ directions;
  • the failures it recorded.

Open two on Compare with Open path A and Open path B.

  • The header names each run, its teachers and its layout.
  • The headline is built from the deltas that stand.
  • Five tiles:
    • ΔF: failure events.
    • ΔTc: convergence timing.
    • ΔTd: structural emergence.
    • ΔĀ: spectrum concentration, λ1/Σλ, how much of the evaluators’ spread falls in one direction. 2.0 called it evaluator agreement.
    • ΔO: stability.
  • The trajectory, with each evaluator’s direction.
  • A panel per evaluator score.
  • Speed, curvature, effective dimension and anisotropy.
  • The eigen spectrum over time.
  • The failures on a shared timeline.
  • A time slider that moves a marker through every view. Nothing animates.

The five rules are 2.0’s, ported and checked against 2.0’s own results. The export contract below changes what they do on checkpoint × item exports. The corrections made in 3.x are listed with their reasons in the spec (decision brief).

A geometry export can state two things about its own layout in run_metadata:

  • step_axis: training_step means the steps are training time. checkpoint_by_item means the steps are blocks, one per checkpoint, over a fixed item order, as in a drift export. Then checkpoints gives the number of blocks.
  • scalar_source: live, replayed (epochs after the first replay cached scores), or fixed_per_item.

When a run’s steps are not time:

  • ΔTc, ΔTd, ΔO and ΔF are withheld, with the reason “steps are checkpoint × item, not time”.
  • ΔĀ is compared checkpoint by checkpoint: each run’s mean λ1/Σλ over a checkpoint’s items, whose order does not matter. It fires only when one run is more concentrated by at least the floor (0.05) at every checkpoint, and then says so: “Path B had a more concentrated spectrum at all 3 checkpoints”. When the sign changes between checkpoints it is quiet. When the two runs have different numbers of checkpoints, it is withheld.
  • The trajectory is drawn as unordered points, one shade per checkpoint.

When a run’s scores repeat:

  • ΔF is withheld, because its failures are score dips.
  • The score panels say so.

In both cases, the “Why” of a withheld tile keeps what 2.0 would have said, and the headline speaks only from the deltas that stand. A file that states neither field, such as a schema 1.0 file, is read exactly as 2.0 read it.

When both runs’ scores are fixed per item, ΔĀ compares the evaluators, not the training runs: their eigenvalues come from the evaluators’ fixed scores, the same for any two runs under the same teachers. The tile says “Evaluator setups differ”, and the headline keeps it apart from what the runs did: “No meaningful divergence between the runs; the evaluator setups differ (spectrum concentration).”

Two real training runs, compared per step and by drift, are in samples/geometry. Their screenshots and a reading of each delta are in the repo’s geometry receipt, recorded with 3.1.0’s rules. On the drift pair, 3.1.1 also withholds ΔTd and reads ΔĀ as a difference between the evaluator setups.

One limit holds even with the contract. In a per-step export, consecutive steps are often different prompts. ΔO can then read the student moving from prompt to prompt as instability. The contract cannot tell the two apart, because those steps really are in training order.

A per-step export follows one training order over sampled dialogues. A different seed samples different dialogues, and the teachers score them differently. So two runs of the same condition with different seeds can fire ΔĀ, on its sustained-stretch rule, and ΔO, on noise alone.

The repo’s test fixtures show it. These are seeds 43 and 44 of aspire-si’s fine-tune-then-ASPIRE run, the same condition at both seeds:

Pair (seed 43 vs seed 44) ΔĀ ΔO
control, local teacher fires: “Path B had a more concentrated spectrum (sustained 12 steps)” fires: Path B shows 1.84 more oscillation
fine-tuned, local teacher fires: sustained 28 steps fires: Path A shows 0.41 more oscillation

The drift exports of the same runs agree between seeds: in all six drift families, seed 43 against seed 44 reads “No meaningful divergence observed between paths”. To compare conditions, use drift exports, which score a fixed set of exchanges, and use more than one seed per condition.