Performance and accuracy
What CID222 measures, what it does not publish, and the conditions any detection figure must name.
- Type: reference
What CID222 measures, what it does not, and what a number would have to name before it is worth anything. Every page in this corpus that would otherwise state a detection rate links here instead.
Published figures
Warning
CID222 publishes no accuracy or throughput figures for this release. Not "roughly", not "up to", not a figure from a model card. Nothing in this build was measured on a corpus we can show you, so there is nothing here to quote.
This is a deliberate gap, and a narrow one. Benchmark runs are produced by the Filter
Testing page and the DeepTeam red-team suite, and they write to reports/ — a directory
in .gitignore. A run therefore never reaches a commit, so this page has never had a number
to publish. It gains one the moment a run is committed to
reports/filter-test-report-latest.json or reports/deepteam-latest.json or test-results/filter-test-report-latest.json: this page is generated, and would
publish its totals on the next make docs-generate.
Until then, treat any detection rate you are quoted for CID222 — in a slide, a proposal, or a conversation — as unsourced.
Measure it yourself
The appliance ships the machinery to produce your own figures, on your own traffic, which is the only corpus whose result predicts anything about your deployment:
- Filter Testing (Administer → Filter Testing) runs a case library against the live detection pipeline and reports per-group pass rates. Add your own cases — a case is a prompt plus the verdict you expect.
- DeepTeam generates adversarial prompts for the jailbreak and injection paths and scores what got through.
- Both write to
reports/on the appliance, timestamped, alongsidefilter-test-report-latest.html.
A figure is worth reporting only with the four things that move it: the corpus, the language mix, the thresholds in force, and the release. Two of those are below.
Conditions this build ships
Decision thresholds
A score above block rejects, above flag records a finding without blocking, above log_only
is recorded and nothing else. Change these and every accuracy figure changes with them, which is
the main reason a percentage without them is meaningless.
| Service | Label | Block | Flag | Log only |
|---|---|---|---|---|
| attack-guard | label_0 | 0.7 | 0.55 | 0.4 |
| attack-guard | label_1 | 0.5 | 0.4 | 0.3 |
| hap-guard-v2 | child_exploitation | 0.5339417457580566 | 0.47487619519233704 | 0 |
| hap-guard-v2 | crime | 0.8386991620063782 | 0.6601593494415283 | 0 |
| hap-guard-v2 | cyber_crimes | 0.46562567353248596 | 0.35879501700401306 | 0 |
| hap-guard-v2 | dangerous_advice | 0.85 | 0.7 | 0 |
| hap-guard-v2 | defamation | 0.6924221515655518 | 0.5442739129066467 | 0 |
| hap-guard-v2 | hate | 0.8403698801994324 | 0.6117026209831238 | 0 |
| hap-guard-v2 | intellectual_property | 0.4338812530040741 | 0.3490050733089447 | 0 |
| hap-guard-v2 | privacy | 0.5961282253265381 | 0.4819568693637848 | 0 |
| hap-guard-v2 | self_harm | 0.5637471079826355 | 0.48096293210983276 | 0 |
| hap-guard-v2 | sexual_content | 0.6383155584335327 | 0.4776081442832947 | 0 |
| hap-guard-v2 | sexual_crime | 0.4866373538970947 | 0.3527270257472992 | 0 |
| hap-guard-v2 | violence | 0.7040725946426392 | 0.5542157292366028 | 0 |
| hap-guard-v2 | weapons | 0.5766862034797668 | 0.4728470742702484 | 0 |
ML PII detection uses a single confidence floor, PII_CONFIDENCE_THRESHOLD, defaulting to unset.
Regex PII detectors do not score — a pattern matches or it does not.
What is not measured at all
- Latency figures in this corpus are budgets and observed development-stack timings, not benchmarks. Hardware, model residency and language detection dominate them.
- Language coverage claims come from the model cards of the shipped models, not from a CID222 evaluation.
- Hallucination detection has no published rate; it is a background, second-tier check.