METR Assessment: Anthropic's Model Progress About Twice as Fast, Roughly 80% Due to AI

What happened

The assessment ran inside Anthropic from August 12 to October 29, 2026 — eleven weeks — with about 2.5 full-time METR staff working on Anthropic devices, reviewing data on pretraining, reinforcement learning, training compute, internal benchmarks and speedup analyses, and interviewing about 15 staff; METR took no payment, and Anthropic provided laptops and free tokens. The engagement is distinct from the independent-evaluator program Anthropic announced on October 15.

The measure is AECI, Anthropic's version of the Epoch Capabilities Index: since October 2025 it has grown at an estimated 1.9 times the earlier rate, with a 95% confidence interval of 1.5 to 2.8 times, implying roughly two years of historical progress in one year — about the distance from Claude 2.1 to Sonnet 4.5; robustness checks put the acceleration between 1.1x and 2.9x. METR says the acceleration was not driven by faster growth in training compute, and staff interviews and surveys are consistent with AI use doubling the pace of capabilities progress; on attribution, METR's subjective estimate is that roughly 80% of the current acceleration comes from AI, while non-AI factors likely mattered more in late 2025.

The other half of the report is redaction: Anthropic had the right to redact any non-public information, and METR says it redacted a majority of the most informative evidence, with a roughly 100-page full report unpublished, so the public has to rely much more on METR's judgment. METR also reviewed the automated R&D section of Anthropic's August 2026 Risk Report, agreeing that risk from models as of July 15 was low, but saying Anthropic's claim not to have crossed the 2x acceleration threshold rests on contestable judgment calls and that the threshold may plausibly have been crossed; the report found no clear bottleneck that would block acceleration over the next 12 months, but AI is not yet able to make highly significant and fully autonomous contributions to the most important parts of model training.

Key facts

1. Publisher and date: METR (Model Evaluation and Threat Research), Nov 5, 2026; the independent assessment is titled 'Independent Assessment of AI R&D Acceleration at Anthropic in October 2026'
2. Method: an eleven-week, high-access evaluation from August 12 to October 29, 2026, with about 2.5 full-time staff and about 15 staff interviews; no payment was taken
3. Headline finding: Anthropic is likely improving its model capabilities on well-specified tasks at around twice the rate of March 2024 to October 2025; the estimate carries high uncertainty, and the true figure could plausibly be closer to 1.3x or 3x
4. Measure: AECI, Anthropic's version of the Epoch Capabilities Index, has grown at 1.9 times the pre-October 2025 rate since then, with a 95% confidence interval of 1.5 to 2.8 times — roughly two years of historical progress in one year, about the distance from Claude 2.1 to Sonnet 4.5
5. Attribution: METR's subjective estimate is that roughly 80% of the current acceleration is due to AI doing R&D; non-AI factors likely mattered more in late 2025; the judgment carries substantial uncertainty
6. Compute ruled out: METR is confident the acceleration was not due to faster growth in training compute; robustness checks put the acceleration between 1.1x and 2.9x
7. Unreleased models: the assessment counted three unreleased models trained since July 2026; the Claude Haiku 5.5 system card reported 1.7x
8. Redactions: Anthropic had the right to redact any non-public information and redacted a majority of the evidence METR found most informative; a roughly 100-page full report was not published
9. Companion review: METR also reviewed the automated R&D section of Anthropic's August 2026 Risk Report, agreeing risk from models as of July 15 was low, but saying Anthropic's claim not to have crossed the 2x acceleration threshold rests on contestable judgment calls and that the threshold may plausibly have been crossed
10. Bottlenecks and limits: no clear bottleneck was found that would block acceleration over the next 12 months; AI is not yet able to make highly significant and fully autonomous contributions to the most important parts of model training, though researchers report signs that such contributions could come soon