Table 1; SI: dictionary learning and coarse partitions
Recovering planted features
At the reported signal-to-background ratio, code-Gram preconditioning increases matched decoder/ground-truth cosine from 0.167 to 0.480 across three paired seeds.
Recovering a known planted dictionary
Loading recorded comparison…
Methods and interpretation
The experiment measures recovery of a known signal planted in real Pythia-410M activations.
Procedure in the paper: Recreate the frozen real-activation sample and planted-signal recipe, train paired decoder updates at the stated normalisation, then evaluate Hungarian-matched absolute cosine and the declared health metrics.
Code and data
v0.1.0 · 58 KB · View source on GitHub ↗
The Pythia-410M deduplicated planted-feature experiment at beta0.5: original WikiText selection, six complete trainings across three paired seeds, and all recovery/health metrics; historical input equivalence is explicitly assumed.
Each standalone package includes code, shared helpers, required small inputs and reference results, with setup and commands in its README. Model weights and public datasets are obtained separately where needed.
Download complete source (v0.1.0) for all experiments and the companion website.