The information geometry of large language models is shared, learned, and controllableCode and data

From predictions to geometry

Each point represents a set of next-token probabilities. The two paths make token A equally likely while changing the other predictions differently.

Activation space

Language model

Input 1: −0.70 (fixed)Input 2: 0.40 (fixed)Input 3: 0.80 (fixed)Input 4: −0.20 (fixed)Input 5: 0.60 (fixed)h1: Fisher–Rao −0.27; Euclidean −0.27h₁token A: Fisher–Rao 33.3%; Euclidean 33.3%token Ah2: Fisher–Rao 0.44; Euclidean 0.44h₂token B: Fisher–Rao 33.3%; Euclidean 33.3%token Bh3: Fisher–Rao −0.02; Euclidean −0.02h₃token C: Fisher–Rao 33.3%; Euclidean 33.3%token C

Prediction geometry

Fisher–RaoEuclidean

A small neural network with three possible next tokens. The experiments below test prediction geometry in language models. About this visualisation →

A local ruler predicts the change in behaviour.

Loading recorded measurements…

Moving away from the resolved subspace has a cost.

Loading recorded measurements…

The language determines the geometry.

Loading recorded measurements…