Figure 2a; SI: identification
Behavioural identification
Held-out profiled loss rises with squared subspace distance on all eight model–condition paths. Deviations from the resolved subspace have steeper loss–distance slopes than matched-random controls.
Loading recorded measurements…
Methods and interpretation
Four Pythia sizes, two conditions and five path points. Results report the range across the two evaluation folds. The reference subspace is estimated from a stronger model.
Procedure in the paper: Construct the fixed reference family and contexts, calibrate each fold, optimise the nuisance parameters at the frozen path points, and score the held-out folds.
Held-out profiling loss
Five recorded points along each path. Read-out deviations are compared with matched random deviations. Each point averages two evaluation folds; their values remain in the download.
The read-out maps internal activations to output scores. A subspace is a set of directions; chordal distance measures separation between two such sets.
Code and data
v0.1.0 · 99 KB · View source on GitHub ↗
The complete four-scale profiled-KL curvature procedure: pinned teacher logits, calibrated rank-32 chart, actual and matched random paths, threshold-driven profile fits and held-out 1,000-draw analysis. Model snapshot equivalence is assumed from unchanged upstream identities.
Each standalone package includes code, shared helpers, required small inputs and reference results, with setup and commands in its README. Model weights and public datasets are obtained separately where needed.
Download complete source (v0.1.0) for all experiments and the companion website.