Run the filter test suite and DeepTeam adversarial benchmarks against your own rules, then read the pass/fail report.

Filter Testing runs stored test prompts through the live input pipeline and reports which ones your rules caught. The DeepTeam red-teaming tab does the same with adversarial prompts, so you can measure how many attack variants get through.

What do I need?

Licence: Any
Role: admin_user

Prerequisites

The appliance runs with CID_ENABLE_LAB=true. On a licensed build this page does not exist.
The filters you want to exercise are Active — the suite runs the live pipeline, not a copy of it.
For DeepTeam runs, the cid-deepteam service is running and reachable from the gateway.
For AI-generated attack prompts, a provider credential the simulator model can use.

Danger

This is a lab surface, hidden on every licensed build. The whole Testing sidebar group is gated on CID_ENABLE_LAB, and the controller carries the same gate: without it every /admin/filter-tests request answers 404, not 403, so a customer never learns the page exists.

Screenshot pendingFilter Testing with the Test cases tab, five statistics tiles and the test run history table/assets/screenshots/filter-testing@0.4.png

Create a test case

Open the page

Select Testing → Filter Testing.

The page opens on Test cases with the statistics tiles Total Test Cases, Passed (7 days), Failed (7 days) and Errors (7 days).

Start a test case

Select Create test.

Create Test Case opens with three input tabs: Form, JSON and ZIP.

Describe the case

On Form, fill **Name ***, **Test Type *** and **Prompt *** — the text that will be sent through the pipeline.

Name is capped at 255 characters and Test Type at 100.

State what you expect

Set **Expected Action *** to Reject, Mask, Flag or Allow, and fill Expected Entities as a comma-separated list, for example SSN, EMAIL, PHONE.

Read the callout below before relying on these: only the detection counts decide pass or fail.

Classify it

Set Severity (Critical, High, Medium, Low), Tags and, if you use suites, Group.

Tags and severity are how you filter the results table later.

Save

Select Create Test Case.

The case is added to the test-case list and is included in the next run of all active cases.

Warning

Only two rules decide the verdict: actual detections below expected_min_detections fails, and any detection at all when expected_max_detections is 0 fails (the false-positive case). Expected Action and Expected Entities are stored and displayed but never compared, so a case can pass with the wrong action.

To load many cases at once, paste an array on the JSON tab, or upload a .zip of JSON files on the ZIP tab — the archive is read in your browser and anything that is not .json is ignored.

Run the suite

Open the run dialog

Select Run Tests.

Run Filter Tests opens.

Scope the run

Optionally set Run Name, pick a Filter Group (or All Groups), and choose All active test cases or Select specific test cases.

Selecting specific cases shows a searchable list with Select All / Deselect All and a {n} test cases selected counter.

Start it

Select Run Tests.

The dialog closes and the run appears in Test Run History; the table polls for progress. Runs execute in the background — the API returns a run id immediately.

Open the result

Select View Details on the finished run.

The dialog shows Pass Rate, Avg Time and the per-case table with Status, Test Case, Severity, Action (expected and actual), Detections, Time and Failure Reason.

Keep a copy

Select Download PDF.

A letterheaded report is rendered and downloads as filter-test-<id>-<date>.pdf. The toast reads PDF downloaded.

Reading a report

Column	What to do with it
Status	Pass, Fail or Error. Error means the case did not complete — the pipeline threw, so the result says nothing about your rules
Severity	Your own classification of the case. It does not affect the verdict
Action	Actual against expected. A mismatch here is not a failure — see the callout above
Detections	The count the pipeline produced, with the expected range beside it
Failure Reason	Expected at least `{n}` detections but got `{m}` (under-detection) or Expected 0 detections (false positive test) but got `{n}`

Under-detection is the failure that matters: it means real traffic of that shape would pass. Over- detection above expected_max_detections is deliberately tolerated and does not fail the case.

Run a DeepTeam benchmark

Open the tab

Select the DeepTeam red-teaming tab.

DeepTeam Service Status reports whether the service is reachable. When it is not, the tab says The DeepTeam red-teaming service is not currently available. and only historical runs are readable.

Configure the attack set

Select Run Benchmark, then choose Vulnerability Categories for Testing Guardrail and Manipulation Techniques.

Techniques that do not apply to the chosen categories are greyed out and dropped if submitted. Some categories use raw curated payloads and take no technique at all.

Size the run

Set Attempts per sub-type, and Test language if you want the Turkish variants.

The dialog shows Estimated total attacks. Turkish translation adds roughly three model calls per unique prompt, cached within the run.

Decide whether to generate new attacks

Leave Expand corpus with AI-generated attacks off to use the static corpus only, or turn it on and pick Provider, Model and AI-generated prompts per sub-type.

The dialog warns that each generated prompt counts against your provider's billing. Generated prompts are deduplicated against the static corpus.

Start and watch

Start the benchmark.

The progress dialog shows {completed} of {total} attacks completed ({percent}%) and warns No progress in {n} seconds. The test may be stalled. when it stops advancing. On completion the toast reads Benchmark completed! {blocked}/{total} blocked ({rate}% detection rate).

Turn the findings into regression tests

Import the run's attacks as test cases so a fixed bypass stays fixed.

The imported cases appear in Test cases and count under Imported Cases on the DeepTeam statistics tiles.

A benchmark result is a detection rate: attacks Blocked against attacks Bypassed. Treat a bypass as a filter gap and reproduce it as a test case before changing a rule.

Fields reference

Create Test Case — Form tab

Field	Required	Rule	What it does
Name *	Yes	Max 255 characters	Identifies the case in the results table
Test Type *	Yes	Max 100 characters	Free-text class, for example `pii-simple` or `jailbreak`. Drives the By Test Type chart
Prompt *	Yes	Free text	The text sent through the live input pipeline
Expected Action *	Yes	`reject`, `mask`, `flag` or `allow`	Displayed against the actual action. Not a pass/fail criterion
Expected Entities	No	Comma-separated types	Displayed only. Not compared
Severity	No	`critical`, `high`, `medium`, `low`	Your own classification
Tags	No	Comma-separated	Filters the results table
Group	No	An existing test group	Assigns the case to a suite
Active	No	Default on	Whether All active test cases includes it

Detection bounds (expected_min_detections, expected_max_detections) are the two fields the verdict is computed from. Set expected_max_detections to 0 for a negative case.

Verify

The run reaches completed in Test Run History with a non-zero Pass Rate and a case count matching what you selected.
Open a failing case and confirm its Failure Reason names the count that was short — that is the rule to fix on Security filters.
Download PDF produces a file. If the renderer is down the request fails with Failed to render filter test PDF and a 503.

Note

Test runs write no event-log row. Nothing about a run appears in Event logs or in All detections — the results live only in the run history on this page.

If it fails

The page is missing from the sidebar, or every request returns 404 — the build is not a lab build. CID_ENABLE_LAB is not true.
Unable to load DeepTeam runs. The backend service may be unavailable. — the gateway cannot reach cid-deepteam. Historical runs still render; new benchmarks do not start.
Failed to start benchmark: followed by the provider's own text — most often the simulator credential is missing or the chosen model is not permitted for this account.
Benchmark polling timed out. The test may still be running in the background. Check the run history for updates. — the dialog gave up watching, not the run. Reopen it from the history table.
Failed to render filter test PDF: … with a 503 — the report-renderer service is not answering.
Every case reports Error — the detection pipeline itself is failing. Check the ML service health before reading anything into the results.

Next steps

Security filters — fix the rule a failing case exposed.
All detections — confirm the fix on real traffic rather than on stored prompts.
ML engine — check the detector itself when every case in a family fails.

Test filters and run red-team benchmarks