ISMS Copilot

Research and data

Compliance AI research and field data

First-party data from building a specialist compliance assistant: framework coverage, a curated ISO 27001 to SOC 2 control-overlap crosswalk, and three evaluation findings. Point-in-time, sourced, and free of any customer data.

Last updated: 2026-07-15. Jurisdiction: international. Figures are first-party and point-in-time; every number links to its primary source below.

Key figures

ISMS Copilot is a specialist compliance assistant. These are the figures we can source, in one place, for anyone writing about compliance automation or AI in GRC.

75+
frameworks with guidance in the library
19
jurisdictions
33 controls
in the ISO 27001 to SOC 2 crosswalk
3
published evaluation findings

Framework coverage

ISMS Copilot publishes compliance guidance across 75+ frameworks and 19 jurisdictions, from ISO 27001 and SOC 2 to NIS 2, DORA, GDPR, and sector and regional regimes. Depth varies by framework: some are backed by deeper curated guidance, others are covered more generally. The list grows regularly, so we state it as a floor rather than a fixed count.

See the frameworks library for the current list, grouped by jurisdiction.

ISO 27001 to SOC 2 control overlap

A recurring buyer question is how much of one framework you get for free with the other. Our hand-verified editorial crosswalk maps a curated set of the highest-value ISO/IEC 27001:2022 Annex A controls to the SOC 2 Trust Services Criteria, with a confidence rating on every entry and a caveat wherever the fit is loose. The figures below describe that curated set, not the overlap between the full frameworks: it is a starting point to orient a mapping exercise, not an authoritative or exhaustive crosswalk, and the two frameworks do not line up one to one.

33
high-value Annex A controls in the curated crosswalk
18 of 33
of those a strong fit to SOC 2
21
distinct SOC 2 criteria touched
93
controls in the full Annex A universe

Explore the full mapping in the ISO 27001 to SOC 2 control mapper. Identifiers are framework-derived; the descriptions and mappings are original editorial content, not the official ISO or AICPA text.

Evaluation field notes

Publishing real evaluation numbers from a production compliance AI is rare. These are three of ours. Read them as what they are: single-run, point-in-time measurements on a specific model and request shape. Two used synthetic, anonymized fixtures; the third reports anonymized aggregate counts from production traffic. No customer content appears in any of them. Each links to the full engineering write-up.

A green metric that hid a real regression

A clause-mapping metric scored a near-perfect 1.0 while the assistant's per-document analysis fell to roughly a third of its standalone depth.

What we measured. Multi-document depth parity: whether each of seven policies gets the same analytical depth when analyzed together as when analyzed alone, mapping each against ISO/IEC 27001:2022 Annex A. Run on an open-weight model served via OpenRouter, over seven synthetic anonymized policies.

0.34
content-volume parity, against a 0.6 pass gate
0.99
clause-mapping row parity (averaged across both samples), the structural metric that stayed green
0 of 7
documents an independent LLM judge rated undiluted
0.22
per-document output-token ratio, grouped versus standalone

Scope. Single internal baseline run, 2026-06-04, on an open-weight model served via OpenRouter. Point-in-time and specific to that model and request shape; it did not reproduce when the same fixtures ran against a frontier model on a direct API. No customer data.

Read the full write-up

When a baseline is too good to measure

A single-turn baseline scored 0.984 and tied 13 of 14 tasks, so a challenger that was never worse posted a 7.1% win rate against a 60% ship gate.

What we measured. A head-to-head gate over 14 hero-flow tasks on a fully synthetic 30-person SaaS fixture with seeded defects, scored by an LLM judge (temperature 0). Baseline was a single model turn on a frontier model; challenger was a heavier multi-step pipeline.

0.984
single-turn baseline mean score across 14 tasks
13 of 14
tasks tied; the challenger was never worse on any task
7.1%
challenger win rate, with ties counting against it, versus a 60% gate
0.78
the baseline's only sub-perfect task, an over-delivery penalty

Scope. One run per system, 2026-06-11, fully synthetic fixture, no customer data. This is a statement about rubric saturation on these tasks, not a verdict of equivalence between the two systems.

Read the full write-up

A safety classifier that flagged an entire domain

A general-purpose moderation classifier flagged 15 messages as harmful over a 17-day window, and every one was a false positive, because compliance users discuss threats for a living.

What we measured. Production moderation output over a 17-day window ending 2026-04-10, in a security and compliance product whose users routinely discuss attackers, exploits, and money laundering. The fix was a two-stage, domain-aware second-pass judge.

15 of 15
moderation flags that were false positives (precision zero)
17 days
production window measured
zero
false-positive rate on those flags after the two-stage fix
23 of 23
confirmed events reconciled to alert emails in the post-fix audit

Scope. One product, one domain, a small sample over a single 17-day window ending 2026-04-10. Framed as our own measurement on our own traffic with a general-purpose moderation classifier; not a general claim about any classifier's quality outside this domain. No customer data.

Read the full write-up

How to read this data

  • First-party and sourced. Every figure comes from our own product, taxonomy, or evaluation suite, and links to its primary source below.
  • Point-in-time, not a benchmark. The evaluation numbers are single runs on a specific model and request shape at a specific date. They describe how a compliance AI behaved, not a comparison of competing products.
  • No customer data. Evaluations used synthetic, anonymized fixtures; production findings report aggregate counts only.
  • Editorial, not authoritative, on the crosswalk. The ISO to SOC 2 overlap is a hand-verified aid, not a conformity statement. Confirm the mapping for your scope with your team and auditor.

Frequently asked questions

What is the ISMS Copilot compliance AI research page?

A single, dated reference page that consolidates first-party data from building and evaluating ISMS Copilot: how many compliance frameworks it covers, how ISO 27001 and SOC 2 controls overlap, and what our evaluation suite has found. Each figure links to its primary source.

How many compliance frameworks does ISMS Copilot cover?

Our resource library publishes compliance guidance across 75+ frameworks and 19 jurisdictions, from ISO 27001 and SOC 2 to NIS 2, DORA, GDPR, and sector and regional regimes. See the frameworks library for the current list.

How much of ISO 27001 and SOC 2 overlap in your crosswalk?

Our crosswalk deliberately covers a curated set of 33 high-value ISO/IEC 27001:2022 Annex A controls, out of the 93 in Annex A, not the whole framework. Within that set, 18 are a strong fit to the SOC 2 Trust Services Criteria and the set touches 21 distinct criteria. Read those as statistics about the curated subset, not a measure of how much the two frameworks overlap overall, which depends on your report scope and your auditor's view. It is an orientation aid, not an authoritative crosswalk.

Are these evaluation numbers benchmarks?

No. Each is a single-run, point-in-time measurement on a specific model and request shape, framed exactly as the source engineering post frames it. They are our own findings about how a compliance AI behaves under evaluation, not a comparative benchmark of different products.

Does any of this use customer data?

No. Every evaluation used synthetic, anonymized fixtures, and the production moderation finding reports aggregate counts only. No customer content appears anywhere on this page or in the underlying work.

How often is this updated?

As new findings and coverage data land. This version was last updated 2026-07-15. The coverage and crosswalk figures recompute automatically from the underlying data; the evaluation findings are dated to the run they describe.

Primary sources

By ISMS Copilot. Framework identifiers are derived from ISO/IEC 27001:2022 and the AICPA Trust Services Criteria; refer to your national standards body and the AICPA for official wording.

Browse all 20 free compliance tools

Ready to do compliance work faster?

Built for speed, accuracy, and audit-ready output.

Try ISMS Copilot