Who Gets to Publish AI Safety Findings? Labs and Evaluators Test the Boundaries

The debate over independent AI safety reviews has moved from principle to paperwork: who gets access to frontier models, who controls an evaluator’s methods and conclusions, and who decides what the public can read. Anthropic’s new embedded-evaluation plan and OpenAI’s published assessment principles both endorse outside scrutiny, but a Reuters report describes private negotiations in which publication rights, confidential information and trade secrets remain contested. The result is not a settled industry rule. It is an emerging negotiation over how independence can work when the evaluator depends on the company for access.

Why the publication question matters

Safety evaluations are meant to test whether a model behaves as claimed, whether safeguards work, and whether a lab has missed important risks. Their value depends on more than technical competence. Readers also need to know whether the evaluator could choose its own tests, see enough evidence, report unwelcome results and explain any limits on publication.

That creates a structural tension. A lab may need to protect source code, unreleased capabilities, personal data, security-sensitive details and information belonging to customers or third parties. An evaluator, meanwhile, may regard those same details as necessary evidence for a safety conclusion. If the lab can veto a report, the review can look like public relations. If the evaluator can disclose everything, a legitimate security or intellectual-property risk may be exposed.

There is also a practical dependency. An embedded evaluator receives access from the company it is assessing and may need company-managed devices, premises or data systems. Independence therefore cannot mean simply having no relationship with a lab. It has to include safeguards around conflicts, access, methods, editing, redaction and retaliation.

What Anthropic announced

On September 18, Anthropic announced a partnership with Faculty, Accenture’s specialist AI business, to conduct embedded evaluation. The stated work includes evaluating and red-teaming models, conducting alignment assessments, and testing model safeguards. Anthropic said Accenture’s experience with enterprise and government AI deployment would inform the safety work. Anthropic’s announcement describes this as part of a broader commitment to place evaluators inside frontier AI companies.

The promise of employee-comparable access

Anthropic says embedded evaluators will work inside AI companies with access comparable to that of an employee. In its account, that vantage point could let reviewers watch models develop during training, follow decisions about building and deployment, speak directly with employees, verify safety commitments and identify blind spots. The company also says evaluators could report incidents and give the public a more informed account of benefits and risks.

This is a wider remit than a conventional external benchmark. A benchmark may measure a defined capability at a particular point in time. An embedded team could examine the surrounding process: what was tested, what was not tested, how a warning was handled, and whether a safeguard worked outside a controlled demonstration. That access could make findings more useful, but it also increases the amount of sensitive information under discussion.

The limits Anthropic itself acknowledges

Anthropic’s announcement is explicitly provisional. It says there are not yet standards for what information embedded evaluators should receive or how they should report what they find. It also says there is no settled system for funding independent evaluation. Anthropic said it would fund Accenture’s work directly for now, while it remains in dialogue with METR and other nonprofit evaluators; it described pooled or government funding as a longer-term possibility. The company’s account says the partnership is non-exclusive and that its approach will evolve.

Anthropic and Accenture each expect to invest at least $1 billion in building capacity in this area over five years. That is a forward-looking expectation, not evidence that either party has already spent $1 billion, and it does not by itself establish how future evaluators will be governed.

What Reuters reported about publication controls

A Reuters report published October 13 says Anthropic and OpenAI have sought significant control over what outside evaluators can publish, citing two people familiar with private discussions. Reuters also said it reviewed a contract whose language could prevent an evaluation firm from publishing confidential information without a company’s consent. Those are reported allegations and documentary findings about negotiations—not proof that every evaluator relationship uses the same terms.

The stakes are particularly sharp for embedded work. Reuters reported that measures under discussion included having lab employees observe embedded teams and preventing an evaluator embedded at one lab from working simultaneously at another. The report said the boundary between trade secrets and information relevant to model risk can itself become a source of argument and redactions.

Reuters also described negotiations as changing quickly. According to the report, people familiar with the matter said the labs were seeking strict editorial control as recently as October 5, while talks with some evaluators appeared to have evolved by October 8. Because the sources were discussing private negotiations, their accounts should be read as attributed claims rather than a universal description of signed policy.

Anthropic’s response to the framing

Anthropic disputed the broad characterization that it seeks significant control over all embedded evaluators’ publications. In comments to Reuters, the company said that, in current conversations with evaluators it had previously worked with, those reviewers control their methods, evaluation criteria, conclusions and what they publish. That is a claim about the arrangements Anthropic described as current, not an independent verification of every contract or negotiation.

Anthropic also told Reuters that, for new evaluator firms, it reserved the right to comment on and approve reports while the reviewers learn how its models operate and establish a working relationship. That distinction is central to the dispute. A lab may view an initial review period as necessary to prevent misunderstandings and protect sensitive information. An evaluator or reader may see approval rights—especially over conclusions—as a material limit on independence.

OpenAI’s stated position

OpenAI told Reuters that it was not seeking approval rights over the full text of evaluator reports. It said its standard practice is to redact confidential information and that evaluators can disclose whether major redactions were made. This position accepts a role for confidentiality review while rejecting, in the company’s description, a general right to approve an entire report.

Reuters also reported on a previous METR and Redwood Research investigation of an incident involving autonomous agents and Hugging Face infrastructure. OpenAI redacted nonpublic information from the report. The researchers said they changed structure, emphasis, clarity and tone in response to OpenAI feedback, but that nothing important to their conclusions was removed. They stood by the findings and praised the access ultimately provided, while acknowledging that concern about discouraging future access influenced drafting and redaction judgments. The episode illustrates why publication rules are not merely legal fine print: editorial feedback, timing and access can shape what an outside reader sees.

What OpenAI’s public principles say

On September 22, OpenAI published principles for effective third-party assessments. The company says assessors should have deep access across training, evaluation and deployment so they can challenge assumptions, identify missed risks and reach their own conclusions about safeguards. It frames third-party assessment as a way to keep labs accountable to safety claims that are independently supported.

The principles call for an agreed scope and clearly defined claims before testing; proportionate access; transparent methods and standards; disclosed conflicts; relevant expertise; security and confidentiality protections; time for remediation; and responsible publication. They say assessors should maintain editorial independence while protecting confidential and intellectual-property information. They also support clear redaction policies under which labs may request removal of sensitive information, while assessors can note when substantive redactions occurred and how those redactions affected the report.

That is a balancing framework rather than a promise of unrestricted disclosure. OpenAI’s principles recognize that incident investigations can involve sensitive internal or third-party data. They propose confidential reporting to appropriate oversight bodies when full public disclosure is not possible, and they emphasize feedback and correction of inaccuracies. In other words, the principles support independence plus confidentiality and correction—not a single formula that resolves every dispute.

What is known—and what is not

Several points are clear from the public record. Anthropic has announced an embedded-evaluation partnership and has acknowledged that access, reporting standards and funding arrangements are still being developed. OpenAI has publicly supported independent assessment and described mechanisms for confidentiality redactions and disclosure of major redactions. Reuters has reported that negotiations over publication rights and contract language remain active and contested.

Several other points are not established. There is no demonstrated, universal standard governing all frontier-lab evaluators. The public material does not show that every Anthropic or OpenAI evaluator receives identical access, or that every report is subject to the same review process. Nor do the announced investment expectations prove that the projected capacity has been built or that the resulting evaluations will be publicly released in full.

The most consequential test will be an outcome, not a promise: whether a credible evaluator can document a serious finding, preserve evidence, explain uncertainty, identify any substantive redaction and publish a conclusion that the lab did not prefer. Until those cases are visible, claims about independence remain partly prospective.

FAQs

What is embedded evaluation?

Embedded evaluation places outside reviewers inside an AI company for some or all of the assessment process. Anthropic says its evaluators may receive employee-comparable access, observe model development, speak with staff and test safeguards. The approach aims to reveal organizational and deployment risks that a short external test could miss, but it makes access, confidentiality and publication rules more important.

Who does Anthropic say controls findings?

Anthropic told Reuters that reviewers it has previously worked with control their methods, evaluation criteria, conclusions and what they publish. It also said it may comment on and approve reports from new evaluator firms while relationships and model understanding are being established. Those are distinct company-stated positions; they should not be collapsed into a claim that all evaluators have the same rights.

Does OpenAI say it can approve an entire report?

No. OpenAI told Reuters it was not seeking approval rights over the full text of reports. It said confidential information may be redacted and that evaluators can disclose whether major redactions were made. Its published principles likewise endorse editorial independence alongside confidentiality protections, correction of inaccuracies and responsible publication.

Does the $1 billion figure mean $1 billion has been spent?

No. Anthropic’s announcement says Anthropic and Accenture each expect to invest at least $1 billion in building capacity over five years. That is a future projection and should not be reported as money already spent, a guaranteed budget, or proof that a mature evaluation system is already operating.

Is there now a final industry-wide policy?

No. The sources describe company announcements, principles and ongoing or recently reported negotiations. Anthropic says standards for access and reporting are unsettled, while Reuters reports disagreement over publication controls. The current arrangements are therefore evolving and contested rather than a universal final policy.

References

Conclusion

Independent AI safety review will be credible only if access and accountability travel together. Labs have a legitimate reason to protect trade secrets and sensitive data, but broad or opaque control over publication can weaken the very scrutiny the reviews are meant to provide. Anthropic’s embedded-evaluation plan and OpenAI’s principles offer different pieces of a possible framework; Reuters’ reporting shows that the hardest details are still being negotiated. For now, readers should distinguish official commitments from private allegations, projected investments from completed work, and publication promises from reports that have actually survived a contentious finding.