Mechanistic interpretability used to be mostly about curiosity and safety. Researchers would open up a neural network, map out circuits, and say “so that’s how it works,” but the outside world did not get a new tool, drug, or test from that understanding. Goodfire’s Alzheimer’s work is exciting because it crosses that line. The same style of techniques used to study language models and vision models were applied to a biological foundation model, and out of its parameters the team pulled a concrete, new diagnostic algorithm for detecting Alzheimer’s from blood.
From “How does this model work?” to “What mechanism does it know?”
Mechanistic interpretability is the practice of turning an AI model from a black box into a system whose internal steps we can describe. In language models, that has meant things like finding “induction heads” that copy patterns, or neurons that track specific syntax. The goal there is to understand the computation inside the model: which features exist, which circuits use them, and how they combine to produce outputs. Historically, that knowledge has mostly stayed inside ML papers and safety discussions rather than creating new, practical tools in other fields.
Goodfire’s project keeps the same core idea but changes the target. The question is not only “how does the model distinguish Alzheimer’s from healthy?” but also “what biological mechanism is the model exploiting?” The model they study, Pleiades, is an epigenomic foundation model built as a hierarchical attention transformer. It is trained on roughly 1.9 trillion DNA and methylation tokens and then adapted to work with cell‑free DNA (cfDNA) fragments from blood, aggregating them with a hierarchical transformer classifier to predict whether a patient has Alzheimer’s disease. That means somewhere in its learned representations there must be a computation that separates Alzheimer’s cfDNA from non‑Alzheimer’s cfDNA. Mechanistic interpretability becomes the tool to find and describe that computation.
The model inside the model: Pleiades as a transformer, and the hidden “Alzheimer’s detector”
Pleiades itself is a transformer‑style foundation model for epigenetic data. At the lowest level, it takes in cfDNA fragment sequences tokenized as nucleotides and methylation marks and processes them with self‑attention to build embeddings for each fragment. These fragment embeddings are then passed into a hierarchical attention transformer that pools across many fragments and differentially methylated regions, ending in a patient‑level embedding that feeds a classifier predicting “Alzheimer’s” or “control”. The end result is a powerful but opaque mapping from billions of noisy cfDNA fragments to a single disease probability.
What Goodfire did is treat this whole system as a “host” model that contains a smaller, implicit algorithm for Alzheimer’s detection. Instead of guessing that algorithm from prior biology, they read it out from the model itself. They froze the cfDNA‑fine‑tuned Pleiades, then interrogated its internal representations around the final sample‑level embeddings. They asked: which features are actually used to drive the Alzheimer’s vs control decision?
To answer that, they used two families of techniques:
Linear probes, which test whether certain biological properties (like fragment length, methylation, genomic region) are linearly decodable from the embeddings, and thus which information Pleiades has available at each layer.
Sparse autoencoders (SAEs) and other unsupervised methods that factor the embedding space into a set of sparse features, each feature ideally representing a coherent concept across many examples.
Together, these tools turn a giant cloud of hidden activations into a structured catalogue: “here are the features this transformer uses, here is how strongly each feature matters for the Alzheimer’s classifier.”
The surprising result is that the dominant features are about cfDNA fragment length. When they decomposed the embeddings with SAEs and examined which features were most influential for the disease classifier, about nine SAE features stood out, and these all primarily encoded fragment length distributions at biologically meaningful sizes. The fragment lengths form a U‑shaped manifold in embedding space, with special emphasis at 147 base pairs (DNA wrapped around a nucleosome core) and 167 base pairs (nucleosome plus linker DNA). Probing and attribution both supported the same story: the learned “Alzheimer’s detector” inside Pleiades is paying exceptional attention to how cfDNA fragments are cut and packaged, not just to where they come from in the genome.
Extracting a new algorithm: from transformer features to a simple Alzheimer’s blood test
The key step is what happens next. Once fragmentomics emerged as the main internal mechanism, the team did not stop at “interesting, fragment length seems important.” They tried to write down the algorithm that Pleiades implicitly learned.
They manually constructed a set of explicit fragmentomics features: summary statistics of fragment length distributions in different genomic regions, peak heights at around 147 bp and 167 bp, ratios between length bands, and related quantities. This created a vector of hand‑engineered numbers that try to capture the same structure Pleiades focused on in its embeddings.
On top of that feature vector, they trained simple logistic regression models. These models are ordinary statistical classifiers, the same family as a regression you might run in Excel, but tuned on thousands of fragments and validated on an independent cohort:
- A model using only fragment length features achieved an AUROC around 0.78 on an external cohort.
- A model combining fragment length with other mechanistically suggested signals, such as methylation and cell‑type related features, reached about 0.84 AUROC with confidence intervals overlapping the full Pleiades cfDNA classifier.
In other words, by following the trail left by the transformer’s internal features, they extracted a compact, human‑readable algorithm: “look at these specific aspects of fragment length and related signals, then linearly combine them to estimate Alzheimer’s probability.” Pleiades acts as the vast search engine over all possible signals in cfDNA; mechanistic interpretability is the key that pulls out the particular rule that works best; logistic regression is the distilled implementation that doctors and statisticians can understand and deploy.
That is the conceptual leap. The field did not only gain another understanding of “what goes on inside a model” but gained an actionable diagnostic rule that can be written down and implemented independently of Pleiades.
Why this matters for Alzheimer’s and beyond
For Alzheimer’s specifically, the extracted algorithm matters because it lowers barriers to testing and makes the biology clearer:
The input is a blood sample with cfDNA, not a spinal tap or expensive imaging.
The mechanism involves fragmentomics, the study of cfDNA fragmentation patterns, which is an area already explored in early cancer detection. Goodfire’s work shows that fragmentomics carries strong signal for neurodegeneration as well and not only for tumors.
The final diagnostic model is small and interpretable. Clinicians can see which fragment length features drive predictions and can reason about how they relate to known pathology such as neuron death, nucleosome positioning, and cell‑type‑specific chromatin states.
In principle, this enables scalable screening tests and more targeted follow‑up for patients at high risk. It also gives basic scientists a new, precise hypothesis: changes in cfDNA fragment length distributions, around particular nucleosome‑scale lengths, are robust biomarkers for Alzheimer’s progression. That hypothesis came not from manual wet‑lab exploration but from reverse‑engineering a transformer trained on trillions of tokens.
The broader implication is methodological. This work shows a template for using mechanistic interpretability as a general tool for scientific discovery:
Train or obtain a large foundation model in some scientific domain, such as epigenomics, protein folding, or climate.
Adapt it to a specific task using a downstream head, like disease classification.
Use mechanistic interpretability techniques (probes, SAEs, attribution, geometry analysis) not just to inspect the model, but to identify which internal features truly drive performance.
Translate those features into explicit, low‑dimensional variables that humans can name and manipulate.
Fit simple, transparent models on those variables, validate them on held‑out data or independent cohorts, and treat them as candidate “extracted algorithms” for how to solve the task.
Feed those algorithms back into the scientific process: new biomarkers, new pathways, or new structural hypotheses.
Pleiades is a transformer at its core: it uses attention to integrate information across billions of DNA tokens and across many fragments per patient. Mechanistic interpretability in this context becomes a way to read out and compress the complex algorithm encoded in its weights into something like “a regression over a dozen fragmentomics features.” That is the conceptual shift: interpretability is no longer only a microscope for AI models, it is becoming a compiler that turns the behavior of large transformers into new, human‑usable algorithms for the real world.