Anthropic Gives Third-Party Evaluator METR Employee-Level Access to Its Models

What happened

Reuters reported on October 15 that Anthropic said on Thursday it has granted a team from nonprofit AI evaluator METR employee-level access to its systems, allowing the group to independently assess its advanced models, safety practices, and risk management.

Six METR employees have been provided company hardware, office space, and access to internal tools and systems broadly comparable to those available to Anthropic's own risk assessment teams, the company said.

METR will be able to publish independent assessments of risks, incidents, and safety practices without Anthropic's prior approval, subject only to limited redactions covering customer data, privileged information, and certain model architecture details.

The announcement follows concerns among researchers and evaluators that leading AI labs could wield too much influence over independent reviews through publication restrictions and control over access to sensitive information, Reuters reported.

Key facts

1. Access scope: Six METR employees receive company hardware, office space, and internal tools and systems broadly comparable to Anthropic's own risk assessment teams
2. Evaluation scope: METR can independently assess Anthropic's advanced models, safety practices, and risk management
3. Independent publication: METR may publish assessments of risks, incidents, and safety practices without prior approval; only limited redactions apply for customer data, privileged information, and certain model architecture details
4. Context: The move responds to researchers' concerns that leading labs could shape independent reviews through publication restrictions and control of sensitive-information access