EuCAIFCon 2026: Sparse autoencoders for a neutrino foundation model

EuCAIFCon 2026

Finding and using interpretable latents in a neutrino foundation model with sparse autoencoders

Raphaël Bonnet-Guerrini^1,2, Johann Ioannou-Nikolaides³, Inar Timiryasov³, Vincenzo Piuri¹

¹Università degli Studi di Milano · ²INFN Milano · ³Niels Bohr Institute

01 · Tool

Polysemanticity and Sparse Autoencoders

XAI fails to explain global behavior

↓

Reverse engineering with Mechanistic interpretability

↓

But neurons are polysemantic

↓

Sparse autoencoder

Sparse dictionary learning network: three polysemantic inputs, five sparse hidden latents, three reconstructed outputs

02 · Promise and limits

What sparse features promise ...

What sparse features promise ... and what they do wrong

Golden Gate Bridge SAE feature activating across languages and images

A single SAE feature recognizes the Golden Gate Bridge across languages and images.

Identify latents

Improve model, Learn from them?

Act on the representation

Amplify, suppress, or steer them.

Sparse ⇒ interpretable ⇒ causal

Readable latent?

Hard to tune

Expensive at large scale

How to find interpretable concepts in Physics foundation models?

03 · Model

Applying an SAE to PolarBERT

Overview: IceCube event to PolarBERT to CLS state, read by a sparse autoencoder and by the downstream direction and uncertainty heads

Input≤127 pulses: time, charge, DOM, auxiliary flag.

PretrainingMasked-DOM prediction + total-charge regression, 100M events.

Backbone8 blocks, \(d_{\rm model}=256\), 8M parameters → CLS state \(h_i\).

Task\(g_{\rm dir}:\mathbb{R}^{256}\to S^2\), scored by \(\Delta\psi\).

04 · Method

SAE interpretability is a hypothesis to test

Concept-validation protocol: association, selectivity, intervention

1 · AssociationLatent 640 fires on bright, clean events. AUROC>t → candidate

2 · SelectivityStill separates after matching charge, pulses, zenith. → validated

3 · InterventionZero it and check the impact

05 · Atlas

A validated atlas of interpretable concepts

Five events ordered by increasing activation of the bright-clean latent

Read-out	Physics role	Evidence
\(z_{bc}\)	bright + clean	AUROC .91 · 4/4 seeds
\(z_{bc}'\)	clean, secondary	controls · 4/4
\(z_{aux}\)	auxiliary activity	AUROC .84 · 3/4
\(z_{depth}\)	detector depth	AUROC .986 · layer 2

An atlas, not an inventoryQuality and brightness localize; morphology stays distributed.

Not one lucky coordinate~194 clean-monotonic latents form a continuous quality–brightness axis.

06 · Diagnosis

The direction head barely uses the atlas

The direction head barely uses the atlas … but the uncertainty head does

Intervention	Direction head	Uncertainty head
remove \(z_{bc}\)	+0.06° null	+0.31 causal
remove clean pair	+0.03° null	+0.32 causal
remove clean family (194)	−0.02° null	+0.78 causal
remove clean core	+0.02° null	+0.37 causal
scale \(z_{bs}\)	V-shaped support	monotone 3/4

InterveneZero or rescale the latent, decode, push through the frozen head.

Control20 firing-rate-matched latents; the effect must clear them, not just zero.

Cross-checkPCA finds the same axis unsupervised; probes set the supervised ceiling; the Jacobian shows where each head is really sensitive.

\[U:\; h\in\mathbb{R}^{256}\;\longrightarrow\;\log\!\left(1+\Delta\psi/\mathrm{deg}\right)\]

Backbone and direction head stay frozen, they both heads read the same \(\mathrm{CLS}\) state.

08 · Take-home message

Take home

For interpretability people:

SAEs are worth it: they let you steer directly in latent space.

For foundation model users:

Check your latent representation!!

For everyone?

Understanding our models might be our future jobs..