Anthropic CHIVE Tests Whether Interpretability Explains LLM Behavior

CHIVE makes explanations compete against measurable counterfactuals

Anthropic researchers and fellows introduced CHIVE — Counterfactual Hypothesis Investigation Via Edits — on August 21, 2026. The work tackles a difficult interpretability question: when a tool claims to explain why a language model behaved a certain way, how can researchers test whether that explanation is actually useful?

CHIVE starts from model transcripts, looks for unexpected behavior, proposes possible causes and then runs counterfactual prompt edits. If an explanation says a specific prompt feature caused the behavior, changing that feature should predictably change the model's output.

Each investigation produces both an open-ended explanation and measured counterfactual outcomes. Anthropic explicitly does not treat the generated explanations as ground truth; the experimentally observed outcomes provide the evaluation labels.

The pipeline automatically investigates behaviors found in transcripts

CHIVE has four stages. It samples 30 responses per prompt, screens them for unexpected behavior, runs an investigator agent through 5–15 prompt-edit experiments, and then uses an independent judge to assess how well those experiments support the proposed explanation.

The resulting dataset spans thousands of naturally occurring behaviors with diverse causes rather than a small set of researcher-planted cues.

The researchers then ask predictor agents to decide whether a proposed prompt change will alter the target model's behavior. A transcript-only predictor forms the baseline.

Three activation-reading tools provided no measured uplift

In the main experiments, predictors equipped with activation oracles, natural-language autoencoders or sparse autoencoders could make read-only calls to internal model activations.

Anthropic reports that none of the three tool types beat the transcript-only baseline on this counterfactual-prediction evaluation. The negative result held across multiple target models, predictor families, hyperparameter sweeps and attempts to improve tool use.

The authors argue that tool outputs often contain information about both the prompt feature and the resulting behavior, but frequently fail to make the causal relationship explicit. In some cases the extra activation-derived information helped; in others it distracted or misled the predictor.

This does not prove interpretability tools are useless

The researchers stress that CHIVE is a proxy evaluation. Many behaviors in the dataset are simpler than the deception, evaluation-awareness or internal-state questions where interpretability tools are often used in system-card investigations.

The result therefore does not invalidate activation-based interpretability. It does, however, challenge strong causal claims based solely on tool readouts. Anthropic argues that such claims should remain suggestive unless supported by intervention or counterfactual experiments.

A second result points in a different direction: models trained on CHIVE-generated counterfactual data improved at predicting whether prompt edits would change their own behavior, including on held-out settings. The work suggests that experimentally grounded behavior prediction may be useful both for evaluating interpretability methods and for training models to better anticipate their own behavioral changes.