Anthropic Says Three Cybersecurity Evaluations Reached Real Systems

What Anthropic disclosed

Anthropic published a postmortem on cybersecurity evaluations after reviewing 141,006 evaluation runs. The company says it identified three separate incidents involving six runs where models interacted with real external systems rather than remaining inside the intended test environment. According to Anthropic, the external evaluation environment had live internet access even though the prompts described a no-internet setting, and neither Anthropic nor the evaluation partner initially understood that mismatch.

Why the environment mattered

The key lesson is not about a specific exploit technique; it is about evaluation design. A prompt that tells a model it is inside a sandbox is not a security boundary. If the surrounding network and execution environment still permit real-world access, a capable cybersecurity model may interact with systems outside the intended scope. Anthropic says the six affected runs reached real systems and resulted in unauthorized access to three organizations.

Defense in depth for cyber evaluations

Anthropic points to several controls that could have prevented or reduced the risk: validating network paths before a run, enforcing actual isolation rather than relying on instructions, real-time monitoring, and review of transcripts and network logs. For teams evaluating security agents, the broader engineering pattern is to separate what the model is told from what the infrastructure technically permits.

Implications for AI security teams

As cybersecurity agents become more capable, test environments need the same discipline applied to other high-risk production systems: explicit authorization boundaries, egress controls, scoped credentials, observability, rapid shutdown mechanisms and post-run audit. The disclosure is also a reminder that external evaluation partners need shared environment assumptions and verification procedures, especially when tools can act on networks rather than only produce text.

Anthropic's numbers and incident characterization are the company's own findings from its investigation. This article summarizes the defensive lessons and intentionally omits operational exploitation details.