From Shubhi K | Product & Market Analysis
Responsible AI Reports Scored: What Microsoft, Google and Anthropic Count
On this page
Anthropic reported 11.4 million banned accounts for the first half of 2026. Microsoft reported more than 450 sensitive use reviews in its latest year. Neither company, and not Google either, publishes a count of AI incidents. That gap is the most useful thing a responsible AI report can teach a buyer, because it shows where the disclosure stops being measurement and becomes narrative.
Key takeaways
- None of the three vendors publishes a count of AI incidents. Microsoft, Google and Anthropic all describe incident processes, and none of the documents reviewed here states how many incidents occurred in a period.
- The annual responsible AI report is often the thinnest document a vendor publishes. Google's hardest numbers sit in its Gemini model cards, not in its 2026 progress report, so a buyer reading only the report misses them.
- Microsoft quantifies its governance machinery, not its outcomes. Its 2026 report counts sensitive use reviews, product reviews, red team operations and people trained, which proves activity rather than results.
- Anthropic scores highest on our eight tests, at 13 of 16. It publishes enforcement counts and per-model evaluation percentages, while its incident row is as empty as everyone else's.
The short answer. A responsible AI report is worth reading for what it counts. Score each vendor on red-team volume, evaluation results, capability thresholds, enforcement counts, incident counts, review volume, third-party testing and certification. Of 16 possible points, Anthropic earns 13, Microsoft 9 and Google 7.
What a responsible AI report is, and what it is not
A responsible AI report is a voluntary document in which a model vendor explains how it governs the systems it builds and sells. No regulator sets its format. No auditor signs it. Each vendor decides what goes in, what gets a number and what gets a paragraph.
That freedom is the whole problem for your procurement team. Two reports can run to similar lengths and contain completely different amounts of checkable information. One can count its reviews, its red team operations and its banned accounts. Another can describe the same activities in confident prose and count none of them.
The three vendors in this comparison also publish different kinds of document. Microsoft publishes an annual Responsible AI Transparency Report, now in its third edition. Google publishes an annual Responsible AI Progress Report plus model cards for each Gemini release. Anthropic publishes a standing Transparency Hub with periodic enforcement data and a model report that it updates per release.
Why the document type changes the score
Comparing three annual reports would be the obvious method. It would also be wrong. Google's annual report and its model cards do different jobs, and the numbers live in the cards. Anthropic does not publish an annual report in the Microsoft sense at all.
So this scorecard reads the public responsible AI disclosure each vendor points buyers toward. For Google that includes the Gemini 3.1 Pro model card. For Anthropic it means the hub pages. For Microsoft it means the 2026 report summary and its coverage. We think that is the fair comparison, and we say where it could cut the other way in the limitations section.
The responsible AI disclosure scorecard: eight tests
Each test asks one question. Does the vendor put a number on this, describe it only in prose, or leave it out? A quantified disclosure scores 2. A prose description scores 1. An absent disclosure, or one we could not find in the sources reviewed, scores 0.
The tests follow what a procurement reviewer actually needs to know. How hard did the vendor attack its own systems? What did its evaluations find? Did any model cross a dangerous capability line? How much misuse did it catch, and what went wrong?
| Test | Microsoft | Anthropic | |
|---|---|---|---|
| 1. Red-team volume | 2: more than 100 operations in 2025 | 1: testing described, no count verified in Google's text | 2: attack counts per model, such as 2 successes in 2,800 attempts |
| 2. Model evaluation results | 1: 20 application and platform cards, report itself in prose | 2: numeric results in the Gemini model card | 2: percentages per model in the model report |
| 3. Dangerous capability threshold status | 0: not found in the summary or coverage we read | 2: Critical Capability Level table per domain | 2: ASL level and threshold scores stated |
| 4. Misuse enforcement counts | 1: 11,000+ security submissions, not AI-specific | 0: not in the documents reviewed | 2: bans, appeals, overturns, NCMEC reports |
| 5. AI incident counts | 1: 13 incident playbooks, no incident count | 0: not found | 1: threat reports describe cases, no count |
| 6. Governance review volume | 2: 450+ sensitive use reviews, about 2,500 product reviews | 1: layered governance described | 1: risk report cadence stated, no volume |
| 7. Third-party testing with results | 1: 18-university alliance named, no results | 1: external parties a stated principle, none named in the card section read | 2: METR finding stated, Gray Swan named |
| 8. Certification or assurance | 1: ISO 42001 on named products | 0: not found in the documents reviewed | 1: ISO 42001 claimed, no certifier or date on the page |
| Total, out of 16 | 9 | 7 | 13 |
Scoring is our own judgement against the stated rubric, applied to sources opened in October 2026. A 0 marked "not found" means absent from what we read, not proven absent from every document the vendor has published.
Microsoft's responsible AI report counts process volume
Microsoft published its third annual Responsible AI Transparency Report on 1 September 2026. The accompanying post by Chief Responsible AI Officer Natasha Crampton frames the year around agentic systems and a rebuilt Responsible AI Standard. That standard now sets requirements separately for models, platform services and applications.
The numbers Microsoft chooses to publish are almost all inputs. Its report summary page says it delivered responsible AI training to nearly 20,000 engineers, policymakers and customers. It says it reviewed more than 100 enacted and proposed laws. It says it engaged over 80 employees from more than 30 teams to redesign its standard.
Trade coverage of the full report adds the larger operational figures. Between July 2025 and June 2026, Microsoft supported more than 450 sensitive use reviews, and more than 30% of those cases involved agentic AI. It also ran nearly 2,500 generative AI product and feature reviews in the same period.
Its AI Red Team conducted more than 100 operations in 2025, according to the same coverage. Microsoft says it has published 20 application and platform cards since January 2026. It also counts 13 cross-functional incident response playbooks.
These are real disclosures, and most vendors publish less. They tell you the review machine is large and busy. They do not tell you what it caught, how many launches it blocked or changed, or what failed after release.
What Microsoft describes without counting
The 2026 report introduces an AI Red Teaming Agent, agent evaluators and a tool called RAMPART that turns red team findings into repeatable tests. It names an External Red Team Alliance of 18 universities across six continents. It names collaboration with the US Center for AI Standards and Innovation and with safety institutes in the UK, Australia and Singapore.
None of those partnerships comes with a published finding in the material we read. A university alliance that tests your systems is a credential. A university alliance whose results you publish is evidence. The second is what we want to see in the 2027 edition.
The one assurance claim is certification. Microsoft says it holds ISO 42001 certification across products including Microsoft 365 Copilot, Foundry and GitHub Copilot. The word "including" matters. If your purchase is a Microsoft AI product outside that list, ask for the certificate scope in writing, and use the guide to what ISO 42001 certification does and does not cover to read it.
Google's responsible AI numbers live in its model cards
Google's 2026 Responsible AI Progress Report arrived in February 2026, introduced by Laurie Richardson and Helen King. The introductory post describes a multi-layered governance approach across the AI lifecycle. It describes testing driven by human expertise and supported by automated adversarial systems.
The post itself carries almost no operational figures. We could not extract the report PDF's text during research, and secondary summaries of it offer a red-team count we could not trace to Google's own words. Under our sourcing rules that figure does not get published here, and Google scores 1 on red-team volume rather than 2.
The Gemini model card is the real disclosure
The picture changes completely once you open the Gemini 3.1 Pro model card, published 19 February 2026. Its frontier safety section scores the model against Critical Capability Levels in five domains. Those are CBRN, cyber, harmful manipulation, machine learning research and misalignment.
Every domain reads "CCL not reached". The card also says the model has reached the alert threshold for cyber, as Gemini 3 Pro did, while staying below the uplift needed for the CCL. That is a specific, falsifiable disclosure, and it is more informative than anything in the annual report's introduction.
The card publishes numbers too. On RE-Bench, a machine learning research benchmark, Gemini 3.1 Pro scored a human-normalised average of 1.27 against 1.04 for Gemini 3 Pro. On harmful manipulation it recorded a maximum odds ratio of 3.6 times, the same as its predecessor.
What Google leaves out
The frontier safety section we read names no external testers. It refers to specialist red teams that sit outside the model development team, which are still Google teams. The Frontier Safety Framework page lists involving external parties as a principle, without naming them in that place.
We found no misuse enforcement counts and no ISO 42001 claim in the documents reviewed. Google publishes extensive product transparency reports elsewhere, so this is a gap in the responsible AI disclosure rather than in Google's reporting overall. For your buyer, the practical point is narrow. If you only read the annual report, you will score Google lower than its model cards deserve.
Anthropic's transparency hub counts enforcement outcomes
Anthropic does not publish an annual responsible AI report of the Microsoft kind. Its Transparency Hub launched in February 2025 and is updated on a rolling basis. The relevant parts are a System Trust and Reporting page with enforcement data, a model report per release, and a voluntary commitments page.
The enforcement data is the most concrete disclosure any of the three vendors makes. For January to June 2026, Anthropic reports 11.4 million banned accounts, 398,000 appeals and 42,000 appeal overturns. It reports 15,079 reports submitted to NCMEC, the US National Center for Missing and Exploited Children.
The numbers you can do arithmetic on
Enforcement counts are valuable because you can test them. Appeals came to roughly 3.5% of bans. Overturns came to about 10.6% of appeals, which is a published error signal for the enforcement system. Those two ratios are our arithmetic on Anthropic's figures, not numbers Anthropic states.
Arithmetic also finds a loose end. The three NCMEC components shown on the page sum to 14,825, which is 254 short of the 15,079 total. The page does not explain the gap. It may be a category not broken out, and it is exactly the kind of question a published number lets you ask.
Anthropic's model report, last updated 2 October 2026, gives each model a standard set of fields. Those include testing methods and results and select safety evaluation summaries. Many of the results are numeric.
Claude Opus 5.5 scored 55.8% on an internal AI research and development test, against an 85% threshold. In a sandbox test, 1.5% of cases resulted in an attempted crossing of a containment boundary. For Claude Fable 5.1, computer-use attack success was 0.07%, or 2 successful attempts out of 2,800.
The hub states that external testing by METR produced findings consistent with Anthropic's own assessment. It names Gray Swan as an external red teaming firm. The commitments page says Anthropic achieved ISO/IEC 42001:2023 for its AI management system, but gives no certifying body or date. Treat that claim as a prompt to request the certificate, not as the certificate.
The blank row: no vendor publishes an AI incident count
Every test on the scorecard has at least one vendor publishing a number, except one. None of the three publishes how many AI incidents occurred in a period, how severe they were, or how long they took to resolve.
Microsoft comes closest in shape. It reports 13 incident response playbooks and more than 11,000 submissions to the Microsoft Security Response Center in 2025. Neither is an incident count. Playbooks measure preparation, and security submissions include reports that are not about AI at all.
Anthropic's threat intelligence reports describe specific misuse cases in narrative form. They are useful, readable and uncounted. Google's annual report, in what we could read, does not attempt the question.
Why the incident row matters most to a buyer
Activity metrics and outcome metrics answer different questions. Activity tells you the vendor has a safety function. Outcomes tell you whether it works. A procurement decision rests on the second, and the second is exactly what is missing.
We do not think this is an accident of format. An incident count is the one number that can get worse year on year in public, and every other metric in these reports only ever goes up. That asymmetry is the strongest reason to put the question in your own questionnaire rather than waiting for a report to answer it.
Where this scorecard is weakest
This scorecard has three soft spots, and a careful reader should weigh them before quoting the totals.
The sources are uneven in tier
Several of Microsoft's larger figures reach this post through trade coverage of the report, not through the report PDF itself, which was too large to retrieve during research. We cite them because they are specific and attributed. They should be upgraded to page references in the primary document. Google's report PDF could not be text-extracted, so its score rests on the introductory post and the model card. If the full report contains counts we missed, Google's score rises.
The rubric rewards one kind of disclosure
The tests favour numbers, and Anthropic publishes the most numbers. A vendor could publish many trivial counts and score well. A vendor could publish one excellent qualitative analysis and score badly. We think counts are the right default for procurement, because they can be checked and compared over time. That is a position, and a fair critic could argue that the best governance work is often not countable.
A second bias deserves stating. Microsoft sells AI products built on several model providers, while Google and Anthropic build frontier models. Tests 2 and 3 suit a model builder. Scored only on tests 4 to 8, the totals narrow to Anthropic 7, Microsoft 6 and Google 2.
A disclosure is not a practice
The scorecard measures what each vendor says, not how safe its systems are. A vendor that publishes less may govern better. Disclosure quality is a reason to trust a vendor's claims more, not proof that the claims describe a safer product.
Using the responsible AI scorecard in procurement
You do not need a vendor's annual report to be good. You need it to tell you which questions remain open, so your questionnaire can close them.
Start from the blank cells. For Microsoft, ask for dangerous capability evaluation results for the underlying models in the product you are buying. For Google, ask for enforcement data and the certification status of the specific service. For Anthropic, ask for the ISO 42001 certificate and its scope. For all three, ask for incident counts by severity for the last four quarters.
Most of this fits inside a structured questionnaire. The NIST AI RMF questionnaire for AI vendor RFPs gives you the governance questions, and the AI agent procurement checklist covers the commercial and operational side. Contractual commitments matter more than published ones, and the comparison of model provider terms shows where the binding promises actually sit.
If your vendor is building agents into your workflows, ask how its red team findings translate into your environment. A vendor's own red team tests its product, not your deployment. The guide to running your own agent red-teaming programme covers the part no vendor report can do for you.
Frequently asked questions
What is a responsible AI report?
A responsible AI report is a voluntary document in which an AI vendor explains how it governs the models and products it builds. It typically covers policies, review processes, red teaming, evaluations and partnerships. No regulator sets its format and no auditor signs it, so the amount of quantified, checkable information varies widely from one vendor to the next.
What does the Microsoft Responsible AI Transparency Report 2026 disclose?
Microsoft's third annual report, published 1 September 2026, focuses on agentic AI and a rebuilt Responsible AI Standard. Its summary page reports responsible AI training for nearly 20,000 people and reviews of more than 100 laws. Coverage of the full report cites more than 450 sensitive use reviews and nearly 2,500 product reviews. It does not publish an incident count.
Does Google publish a responsible AI report?
Yes. Google published its 2026 Responsible AI Progress Report in February 2026. The introductory post is largely descriptive. Google's most specific disclosures sit in its Gemini model cards, which score each model against Critical Capability Levels in five risk domains and report that Gemini 3.1 Pro reached the cyber alert threshold without reaching the critical level.
What is in Anthropic's transparency report?
Anthropic publishes a Transparency Hub rather than an annual report. For January to June 2026 it reports 11.4 million banned accounts, 398,000 appeals, 42,000 overturns and 15,079 reports to NCMEC. Its model report gives per-model evaluation results, such as attack success rates and scores against internal capability thresholds, and names METR and Gray Swan as external testers.
Which AI vendors publish AI incident counts?
None of Microsoft, Google or Anthropic publishes a count of AI incidents in the responsible AI disclosures reviewed for this comparison. Microsoft counts incident playbooks and security submissions. Anthropic describes misuse cases in threat intelligence reports. If incident data matters to your decision, ask for counts by severity and quarter in your vendor questionnaire.
How should procurement teams evaluate an AI transparency report?
Separate what the report counts from what it only describes. Check whether red teaming, evaluations, capability thresholds, enforcement, incidents, review volume, third-party testing and certification each come with a number. Note the reporting period of every figure. Then turn every blank into a written question in your vendor questionnaire, and ask for contractual commitments where possible.
Where to start this week
Take your two largest AI vendors and fill in the eight-row scorecard for each, using only what they publish. It takes about an hour per vendor. Every 0 and every 1 becomes a question in your next renewal conversation.
Then send one question to each vendor, in writing, before the end of the month. Ask how many AI incidents affecting customers it recorded in the last four quarters, and how it defines one. The answer, or the refusal, will tell you more than any report has so far.
Related in this series
Regulation is starting to make some of this mandatory. The EU AI Act transparency checklist maps which disclosures become obligations, and the SEC AI-washing enforcement log shows what happens when public AI claims outrun the evidence.
References
- Microsoft, Natasha Crampton, Responsible AI in 2026: How we are adapting for what's ahead, 1 September 2026. Used for report framing, External Red Team Alliance and ISO 42001.
- Microsoft, Responsible AI Transparency Report summary page, 2026. Used for training, law review and standard redesign figures.
- Mexico Business News, Microsoft prioritizes AI governance as agentic risks expand, 18 September 2026. Used for sensitive use reviews, product reviews, red team operations, cards, playbooks and MSRC figures.
- Google, Laurie Richardson and Helen King, Our 2026 Responsible AI Progress Report, February 2026. Used for Google report framing.
- Google DeepMind, Gemini 3.1 Pro model card, 19 February 2026. Used for Critical Capability Level results, cyber alert threshold, RE-Bench and manipulation figures.
- Anthropic, System Trust and Reporting, updated 23 July 2026. Used for banned accounts, appeals, overturns and NCMEC reports.
- Anthropic, Transparency Hub and model report, updated 2 October 2026, and Voluntary Commitments, updated 23 July 2026. Used for evaluation results, METR, Gray Swan and ISO 42001.
Weakest point in the source base: Microsoft's operational figures are cited through trade coverage rather than the report PDF, and Google's report PDF was not text-extracted. Scores reflect documents accessed in October 2026 and will change as vendors publish.
Related reading