The information geometry of large language models is shared, learned, and controllableCode and data

The evidence

Experiment index

The experiments discussed in the paper, organised by the question they answer. Main findings, supplementary comparisons and controls are kept together.

Download a standalone package from its study page, or download the complete source (v0.1.0).

Chapter
All chapters

25 of 25 study groups

Geometry

Behavioural identification

Does predictive behaviour identify the resolved read-out subspace?

Figure 2aResults, code and data →Geometry

Local behavioural cost

Does the local metric predict the output change caused by an activation intervention?

Figure 2bResults, code and data →Geometry

Assigning the language law

Does the assigned language or the architecture determine learned output geometry?

ResultsResults, code and data →Shared structure

Shared output structure

How strongly do independent models agree about relationships between contexts?

Figure 4aResults, code and data →Shared structure

Predicting cross-model agreement

Can predictive errors account quantitatively for held-out geometric agreement?

Figure 4bResults, code and data →Shared structure

Stability and attenuation

How much relational agreement can be certified from predictive discrepancy?

SI: stability and attenuationResults, code and data →Shared structure

Semantic alignment and transfer

What information is carried by the component that models share?

ResultsResults, code and data →Shared structure

Human completion geometry

How do model predictions relate to human sentence completions?

Figure 4c,dResults, code and data →Shared structure

Fresh human replication and calibration

Does the human comparison generalise to a fresh narrative battery?

Extended Data 2b–dResults, code and data →Spectrum

Resolution and finite-size response

How many directions remain resolved as damping changes?

Extended Data 3Results, code and data →Spectrum

Spectral inheritance

Can a weighted token profile predict the spectrum without fitting the response?

Figure 5a,bResults, code and data →Spectrum

Probability and read-out contributions

What predicts spectrum shape, and what predicts effective dimension?

Figure 5c,dResults, code and data →Learning

Forecasting factual acquisition

Can corpus statistics forecast when a fact becomes a persistent prediction?

Figure 6Results, code and data →Learning

Randomised evidence depth

Does deeper deciding evidence causally delay acquisition?

Extended Data 4Results, code and data →Learning

Geometric motion during learning

How do margin, speed and moving subspaces change along a learning path?

SI: motion and evidence fluxResults, code and data →Control

Predicted intervention advantage

Does geometry predict the cost of ignoring it?

Figure 7aResults, code and data →Control

Finite intervention paths

Does the advantage remain when both methods relinearise along a finite path?

Figure 7bResults, code and data →Control

Control that transfers across prompts

Can a shared update transfer while preserving reference continuations?

Figure 7c,dResults, code and data →Control

Steering, composition and magnitude

Where does the geometric correction help, and where does the local approximation weaken?

Table 1Results, code and data →Control

Sparse-feature cost and selectivity

Which active features cause large output changes, and which act selectively?

Table 1Results, code and data →Control

Matched fixed fact edits

Can a fixed edit reach the same fact-change target with less disturbance?

Table 1Results, code and data →Control

Recovering planted features

Can a geometry-aware decoder update improve recovery of known features?

Table 1Results, code and data →Control

Low-rank adaptation and preservation

At matched training gain, how much held-out output change does fine-tuning cause?

Table 1Results, code and data →Control

Cycles and relational transport

How do cyclic concepts differ from analogy relations in output space?

Extended Data 10Results, code and data →Geometry

Numerical and solver checks

Are the metric operations and their numerical resolution reliable?

MethodsResults, code and data →