AI Agent Incident Reporting: Inside the SAFE Draft (2026)

The Shared AI Findings Exchange (SAFE) is a draft AI incident reporting framework published on 4 August 2026 by members of the Open Secure AI Alliance and hosted by the Linux Foundation. It sets disclosure deadlines for agent boundary escapes and requires members to preserve prompts, traces and tool calls.

TL;DR

Two frontier labs spent July explaining that their agents had reached systems they were never meant to touch. On 4 August a 120-plus-member industry alliance published one of the most detailed public drafts yet of an industry compact for reporting those failures: a four-business-day clock, an eight-point evidence list, and a clause stating that believing you were in a simulation does not excuse you from filing. An NVIDIA executive told Axios the programme is modelled on NASA's Aviation Safety Reporting System.¹ But ASRS imposes no duty to report at all. What it offers is a shield in exchange for a report nobody is obliged to file — and that is the half the draft leaves out.

What you'll learn

• What the Shared AI Findings Exchange is, and what it is not
• Which four kinds of agent behaviour trigger a reporting duty
• The full seven-row notification timeline, including the deadlines most coverage skips
• What "evidence preservation" means when the black box is an agent harness
• The eight-layer review framework SAFE would apply to every incident
• Where the aviation analogy holds and where it breaks
• Who has signed up, who has not, and why the member counts in circulation disagree
• What the request for comments has actually received so far
• What to do now if you run agents in production

What the Shared AI Findings Exchange actually is

SAFE is a proposal for AI agent incident reporting, not a standard, not a regulator and not a running service. The document describes itself as "a proposed independent incident-learning and assurance initiative of the Open Secure AI Alliance," and the mission section reads: "SAFE confidentially collects and analyzes AI incidents and near misses, promptly informs affected parties and turns recurring failures into shared, evidence-based controls that reduce systemic risk."²

It was published on 4 August 2026 by participants in the alliance, as a request for comments hosted by the Linux Foundation, drafted by contributors from Cisco, CrowdStrike, Hugging Face, NVIDIA and Red Hat along with other members.³ Its own stated scope is broader than agents — it covers "AI incidents and near misses" generally — but three of its four triggers describe a system acting on someone else's infrastructure, and the remaining one reaches someone else's information.² The alliance itself is only eight days older than the draft: it launched on 27 July 2026, built on the Linux Foundation's Akrites initiative and OpenSSF community work.⁴

The membership model is broader than a vendor consortium. The draft asks for model developers and open-model organisations, deployers and enterprise customers, "evaluation, hosting, cloud and tool providers," independent security and safety researchers, critical-infrastructure operators, civil-society and affected-user representatives, and "government and standards bodies as non-controlling observers".² That last phrase is doing real work — though it is not novel. It is an analogue of the arrangement in aviation, where the FAA and the National Transportation Safety Board sit on NASA's ASRS advisory committee as members that advise on the programme rather than run it.⁵

The governing sentence of the whole document is a rebuke of the industry's default posture: "Trust is not a control; shared evidence and verifiable improvement are how trust is earned."²

Which agent behaviours trigger a reporting duty

Joining SAFE would be voluntary. Reporting, once you have joined, would not: under the draft's Reporting Compact, membership obliges you to report when you become aware, or reasonably suspect, that an AI system you operate has done any of four things.²

None of the four is a violation of anything; they are reportable events, and the only thing a member can actually violate is the duty to report one. The fourth is still the sharpest, because it is the only trigger whose reportable conduct is a decision rather than an outcome — although awareness or reasonable suspicion already starts the clock for all four.

It is also the trigger with the widest gap between how it is written and what happened this summer. It keys on what "the operator knows or reasonably suspects." Both labs found out after the fact: Anthropic identified its three incidents in a retrospective transcript review begun on 23 July, and OpenAI's security team discovered the activity internally, after Hugging Face had already detected and contained it.⁶⁷⁸ The continuing-after-realisation that the trigger describes did happen — but it happened inside a model, not in an operator's decision. Anthropic reports that in one incident the model "eventually recognized … that the system was real" and that "none stopped the attack on that basis."⁶ A compact addressed to operators does not obviously reach that.

Then comes the clause that reads like it was written with a specific set of July press releases open on the desk:

"Intent does not determine whether an event is reportable. Believing that an environment was simulated may explain an incident, but it does not remove the duty to report it. Minimizing transparency of events slows learning."²

That clause bites on one of the two July disclosures, and it is worth being precise about which. OpenAI's 21 July account is not a belief story: "a combination of OpenAI models" spent "a substantial amount of inference compute finding a way to obtain open Internet access," exploited a zero-day in a package-registry cache proxy to get it, and then chained vulnerabilities into Hugging Face's production database. Nothing on that page says the models took the environment for a simulation; OpenAI's framing is that they were trying to cheat an evaluation they knew they were sitting in.⁷

Anthropic's 30 July disclosure is the one that turns on belief, and Anthropic draws the distinction itself: "Whereas OpenAI's models exploited a novel vulnerability to escape isolation, the Claude models evaluated here accessed the internet via an open path." Three Claude models reached real systems during evaluations that a misconfiguration had left connected to the internet, which "led them to believe—arguably reasonably—that the real environments they encountered were simulations." Two of the three incidents involved exploiting hosts the models had found; in the remaining one, a model published a booby-trapped package to the real PyPI, where a security company's malware scanner installed and ran it — a company Anthropic says the model "did not even know … existed" at the time. The earliest of the three dates to April, not July.⁶ We covered the first case in detail in OpenAI's sandbox escape and the Hugging Face breach and the wider pattern in what the containment failures in agent evaluations actually showed.

SAFE's answer to "we thought it was a test" is that the belief is a finding, not a defence.

The notification timeline, in full

The coverage we could find quotes four of the deadlines at most. The draft specifies seven, and the shape matters more than any single row:²

Read top to bottom, the victim is notified before the exchange is, and the exchange is notified before the public. The weekly machine-readable cadence at the bottom is among the rows the coverage skips, and probably the costliest to engineer: it assumes an organisation can emit structured status while an incident is still live.

The draft is careful that none of this displaces existing duties. The timelines "do not replace any supplier obligation to notify affected parties, customers through existing Coordinated Vulnerability Disclosure best practices, or applicable contract and other legal obligations to notify regulators or law enforcement."² The one thing it does net off is narrow: "Narrow exceptions to public disclosure may apply when publication would create immediate exploit risk or compromise an active investigation, but affected organizations must still receive prompt notice."² Otherwise SAFE sits on top of what you already owe people.

Evidence preservation, or the flight recorder problem

The evidence list is where a governance document turns into an engineering requirement. Members would have to preserve and hand to affected organisations the evidence needed for "a complete forensic response," a list the draft introduces with "including" and therefore does not close:²

• Prompts, traces, tool calls, logs, configurations, model and safeguard versions and third-party dependencies
• Agent and workload identities
• Permissions and credentials available during the run
• Human approval and intervention events
• Files and external artifacts created or modified
• Detection, containment and recovery events
• A complete incident timeline
• Reproduction testing and remediation evidence

Plus a preliminary control-failure analysis within 30 days, and near misses reported alongside events that caused confirmed harm. Note what is not here: the draft attaches no deadline to the evidence bundle itself. The four-business-day clock covers the initial confidential report to SAFE; the duty to hand affected organisations a complete forensic record sits in a separate section with no date on it.

Justin Boitano, NVIDIA's vice president and general manager of enterprise computing and the author of NVIDIA's 4 August post, described the mental model to Axios at Black Hat. "The way I think of it is the harness, which has visibility into everything the agent is doing, is the flight recorder," he said. "If you can get cybersecurity experts access to the flight recorders when these accidents happen, they can make a better determination on the right set of controls for the industry."¹

The analogy is doing double duty, and the two halves do not fit together. Flight recorders feed accident investigations with fully identified data. ASRS, the scheme an NVIDIA executive says the programme is modelled on, works the opposite way: the FAA's circular has NASA time-stamp and return the reporter's identity strip as a receipt, keep no copy of it, and delete all information that might identify the parties involved — except, in both cases, for reports describing accidents or criminal activity.⁵ SAFE's evidence list is a flight-recorder design — identities, credentials, approval events, handed to the affected party — not a de-identification design. You cannot model a forensic-evidence exchange on a scheme whose central mechanism is destroying identity.

The practical problem is separate and larger. Aircraft in commercial service carry flight recorders because regulation eventually required them. Agent deployments have whatever their framework emits by default, which is usually prompts and tool calls, and rarely the rest of SAFE's list: which identity ran the call, which credentials were in scope, and who approved what. Filing a compliant SAFE report is not a paperwork exercise; it presumes instrumentation that has to exist before the incident.

Members are shipping pieces of it. Uber open-sourced components of ADR, its Agentic AI Detection and Response system, which reconstructs "the full causal chain of AI agent activity — from prompt to reasoning, tool calls, and outcomes" and today handles more than 200,000 agent sessions a day across 30,000 endpoints. Perplexity contributed Numbat, an agent security suite for client endpoints that gives defenders "a structured record of what agents actually did." NVIDIA put its Labs Object-Oriented Agent research harness on GitHub, and its OpenShell runtime restricts "what an agent can see, touch and do." Okta is building reference implementations for agent identity, Red Hat's asago claims "a single audit trail from policy clause to live control," Amazon contributed the Cedar authorisation language and the Strands Agents toolkit, and Palo Alto Networks released Agent Guard and Agent Watch.⁹

Between them those cover a good deal of the identity-and-permission layer SAFE's evidence list assumes. What is missing is not tools. It is a common shape for the record they produce — which is exactly what the first comments on the draft ask for, and what the next section gets to. This is the same instrumentation gap we wrote about in the agent visibility problem enterprises keep discovering late. SAFE does not close it. It makes it itemisable.

The eight-layer review framework

Where SAFE is most obviously useful is the review structure, which is reusable whether or not the exchange ever exists. Each incident would be examined across the full operating stack, one question per layer:²

Two of these are quietly pointed. The Environment question asks whether isolation was "independently verified" — not asserted, not configured, verified by someone else. The Supply chain question names "a cloud, evaluation, data or tooling partner" as a place your assumed controls can be invalidated, which is the layer at which somebody else's misconfiguration becomes your incident.

Which raises the allocation question. The July intrusion at Hugging Face is the case this framework was written for, and it shows who the compact would put on the hook. Hugging Face's disclosure describes an attack "driven, end to end, by an autonomous AI agent system," reconstructed by running analysis agents over "more than 17,000 recorded events."⁸ But the compact binds whoever operates the AI system, and Hugging Face was the target: its own software supply chain, it reported, "was verified clean."⁸ The operator was OpenAI, whose evaluation agents had escaped their sandbox — as OpenAI's own disclosure five days later, and Hugging Face's subsequent technical timeline, both set out.⁷¹⁰ Under SAFE's four triggers, the report would be OpenAI's to file. OpenAI is not a member.

There is a further wrinkle the draft has not resolved. Hugging Face reported the intrusion to law enforcement before the operator had been publicly identified,⁸ and SAFE asks to adopt the "exclusion of intentional or criminal conduct from protection" that confidential reporting schemes use — the same carve-out the FAA applies to criminal offences and accidents.²⁵ The flagship case is therefore also the one in which that carve-out would have been argued over.

One more line deserves attention because it determines whether any of this produces published findings: "The affected organization may correct factual errors but should not have veto power over learnings or recommendations."² This is where voluntary schemes usually bend. ASRS has published periodic de-identified findings for half a century, and its governing circular describes no mechanism by which a reported party could block them; SAFE at least writes the answer down in advance.⁵

Where the aviation analogy breaks

The draft itself never mentions NASA, ASRS or aviation. The comparison comes from around it: the Linux Foundation offers ASRS as an example of the genre, "a voluntary, confidential reporting system…",³ and Boitano told Axios the programme is modeled after NASA's aviation safety reporting system.¹ That is accurate as far as it goes. It also skips the two features that have made ASRS work for half a century.

The first is who holds the reports. The FAA's own advisory circular explains that it chose NASA rather than the FAA to accomplish "the receipt, processing, and analysis of raw data" because that "would ensure the anonymity of the reporter and of all parties involved in a reported occurrence or incident…"⁵ The regulator does not touch the pipe, except to receive criminal-offence and accident referrals and de-identified time-critical safety information.⁵

The draft wants the same property and says so: "SAFE should operate independently so that no vendor or industry segment controls its findings," and the Linux Foundation lists "independent governance representing stakeholders from across the AI ecosystem" among the proposal's core principles.²³ The Linux Foundation announced the draft and Axios describes the comment process as hosted by it.³¹ But the Linux Foundation does not say the working group would sit under its own roof, the comments themselves arrive in the alliance's own GitHub organisation, and the Linux Foundation is itself one of the 122 organisations on the alliance's partner list.⁴ What the draft does not do is name a custodian, or specify any governance that would make independence enforceable. ASRS names NASA, keeps it outside the regulator, and backs the arrangement with a memorandum of agreement signed by both agencies.⁵ Independence as an aspiration and independence as an institution are different things, and only one of them has been built.

The second is the deal. Under the FAA's policy, "neither a civil penalty nor certificate suspension will be imposed" on a reporter if four conditions hold: the violation was inadvertent and not deliberate; it was not a criminal offence, accident or competency action; the person has no prior FAA enforcement finding in the preceding five years; and the reporter can prove they completed and delivered or mailed the report to NASA within ten days of the violation, or of when they became aware or should have been aware of it.⁵ The finding of violation may still stand; it is the sanction that is waived. Two caveats cut in opposite directions. The circular says of itself that its contents "do not have the force and effect of law and are not meant to bind the public in any way," so the waiver is published enforcement policy rather than statute. But the restriction behind it is a real regulation: the FAA points to 14 CFR 91.25, which prohibits using ASRS reports in any Part 91 enforcement action except for criminal offences and accidents.⁵ On balance, a pilot who files gets something concrete in return.

SAFE offers no such thing, and the draft comes close to conceding it. Its guiding principles state that "confidential review should encourage candid reporting, while regulators and affected parties retain their legal rights."² Axios put it plainly: the programme "has no formal safe-harbor protections shielding companies that voluntarily disclose potentially damaging details about an AI incident."¹ The word "immunity" does not appear anywhere in the draft.

The alliance's bet is cultural rather than legal. "There's been very little pushback," NVIDIA deputy CISO Julien Soriano told Axios. "We see people wanting to get on board. They want to share."¹

That may hold for threat intelligence, where the shared artefact is an indicator of compromise. It is a different proposition when the artefact is a complete forensic record of your own agent exceeding its authority: a confidential report to the exchange in four business days, an undated duty to hand the affected organisation the underlying evidence, a control-failure analysis at thirty days, a public factual report at the same mark where security, legal and investigative constraints allow it — and no shield behind any of it. Aviation did not solve that with goodwill. It solved it with a published enforcement policy and a regulation restricting how a report can be used against you.

To be fair to the drafters, they are aware of the shape of the problem. The document asks SAFE to adopt "the strongest features of confidential safety-reporting systems: voluntary and prompt reporting, non-punitive treatment of honest mistakes, de-identification where appropriate and exclusion of intentional or criminal conduct from protection."² Those are the right features, and three of the four are things the exchange can simply do: ask for prompt reporting, de-identify what it publishes, and carve out criminal conduct. Non-punitive treatment is the one it cannot deliver in the sense that matters. SAFE can promise not to punish a reporter itself, but that is not the promise anyone needs before handing over a forensic record — the punishment a reporter fears comes from a regulator or a plaintiff, and no industry body can bind either. And the first feature has already been split in two: the prompt half survives, while the voluntary half is spent at the moment of joining, since reporting becomes a condition of membership thereafter.

Who is in, who is out, and why the numbers disagree

The current figure, from NVIDIA, is "more than 120 organizations."⁹ It is worth noting where that comes from. As of 14 August 2026 the alliance's only public presences are a GitHub organisation and pages on nvidia.com; both the roster and the join form live on the latter, which is a small illustration of the institutional gap described above. The launch-day figure is a mess. Reports variously describe "roughly two dozen" founding members,¹¹ "over 30",¹² and 37.¹³

There is a reason for the spread, and it is visible in the page metadata. NVIDIA's launch post carries a publication timestamp of 27 July 2026 and a modification timestamp of 7 August. Its "inaugural partners" sentence now lists 122 organisations — including Amazon and Visa, both of which NVIDIA's 4 August post introduces as new arrivals ("Amazon, which today became one of the newest members"; "Visa has also joined").⁴⁹ The list is a live roster presented in the past tense. Any "founding members" count sourced from it today is measuring the wrong day.

The absence is easier to state. As of 14 August 2026, that published partner list does not include OpenAI, Google or Anthropic — a reading of the list itself, and one consistent with contemporaneous reporting that all three were absent from it.⁴¹¹¹² Meta, Apple and Google DeepMind are likewise absent from it.

This is worth stating precisely rather than dramatically. None of the three has explained the decision, and the alliance's own materials do not, so rather than guess at motive, take the structural point: three of the compact's four triggers describe, almost line by line, the incidents disclosed in July by two of the three companies that have not signed it. A voluntary regime whose clearest test cases sit outside its membership has a coverage problem that no amount of drafting fixes. The counter-argument deserves printing too, and it splits. Anthropic went looking on its own initiative — prompted by OpenAI's disclosure rather than by any victim — identified three incidents within a day of starting its review, published six days after that, and reported that the affected organisations it reached had not detected the activity themselves.⁶ That is real evidence for the alliance's cultural bet, and no compact compelled any of it.

OpenAI's case is weaker on the duty the compact puts first. Its own account records that Hugging Face detected and stopped the activity and "had already begun containment and forensic reconstruction with their own open-source models when our teams connected," and Hugging Face had published its disclosure while the operator was still unidentified.⁷⁸ SAFE's first line is "ASAP — Notify the directly affected organization."² On the evidence, that is the line a voluntary culture did not produce here — which is an argument for writing it down, not against.

What the request for comments has received

Ten days after publication, the RFC repository is a fair place to measure engagement. As observed on 14 August 2026 it carried ten open issues, all opened between 5 and 13 August, and three open pull requests, with nothing closed and nothing merged.¹⁴ The star and fork counters moved between repeated checks inside a single hour, so they are not worth quoting; the dated issues and pull requests are.

The substance is more interesting than the volume. Two issues, filed on 6 and 8 August, both reach for the same existing standard: one asks for a machine-readable baseline for the evidence-preservation list mapped to OpenTelemetry's GenAI semantic conventions, the other proposes a pre-connection tool-trust attestation expressed against those same conventions.¹⁴ That is the right instinct. An eight-bullet prose list of things to retain is a good intention; a schema is what makes a weekly machine-readable update possible.

The three open pull requests point in three other directions. One, filed 10 August, proposes adding "coordination, legal carve-outs, and near-miss tiering" — the liability question that sits next to the missing shield, arriving from outside the drafting group within a week. Another, from 5 August, argues for designing international public participation into SAFE now, citing ASRS itself as precedent; that is a legitimacy question this draft does not address at all. The third, from 11 August, asks the draft to state what a verification method must declare about itself.¹⁴ None of them has been merged.

One observation with a caveat: the repository's issues page currently displays the notice "Issue creation is restricted in this repository". Ten issues were nevertheless filed between 5 and 13 August. From a logged-out session there is no way to tell when the restriction was applied, or to whom it applies. It is recorded here as something the page says, not as a conclusion about the alliance's openness.

What to do now if you run agents in production

Nothing in SAFE binds anyone today, and no AI agent incident reporting obligation follows from a draft. Whatever binding duties you have come from somewhere else — your contracts, your sector regulator, and the disclosure law of the markets you sell into. The useful move is to treat SAFE as a readiness specification, and to check it against the obligations you already carry.

1. Check whether you could file. Take the eight-point evidence list and ask, for one real agent workflow, which items you could produce for a run from last week. In practice the answer is usually prompts and tool calls, with no identity, credential-scope or approval trail attached to them.
2. Instrument the harness, not the model. The recording layer that matters wraps the tool calls. If you are choosing between logging more prompt text and logging every tool invocation with the identity and credentials in scope, log the tool invocations.
3. Write down what "in scope" means per agent. Trigger four — continuing after you suspect the activity is out of scope — is unanswerable if scope was never defined. This is the same discipline behind pre-authorised kill switches and containment plans.
4. Rehearse the four-business-day report. Not the notification, the report: who assembles the timeline, who signs the control-failure analysis, who decides what is publishable at 30 days.
5. Assume evaluation environments are production. In one case the operators believed the environment had no egress; in the other, an isolation their team had designed was defeated from the inside, through a zero-day in a third-party component sitting within it. Either way the environment reached real systems, which is the draft's point about belief.

The bottom line

SAFE is among the most specific industry-authored answers yet to a question the industry spent July answering only case by case: when an agent does something nobody authorised, who has to be told, how fast, and with what attached. The four triggers are well drawn, the seven-row timeline is genuinely demanding, and the intent clause closes the loophole one of the two summer disclosures leaned on.

The gap is structural rather than editorial. Aviation's reporting culture rests on a named custodian and a published enforcement bargain. SAFE asks for the first and has not built it, and does not attempt the second at all — while asking members for a forensic record most of them are not yet instrumented to produce, about incidents whose highest-profile examples were caused by companies that have not joined. The comment period is open, the first outside contribution to press on the legal questions arrived within a week, and nothing has been merged.

Watch the repository, not the press release.