Ask the AI Analyst
Question your own detection data in plain language. The analyst runs on a local model, reads through read-only tools, and withholds any answer it cannot verify.
- Version: 0.4
- Feature: risk.llm_review
- Role: admin_user
- Type: task
AI Analyst is a chat over your own data. You ask a question in your own words; a local model looks the answer up through read-only tools, draws the rows it found as a table, and shows you which lookups it made. An answer whose figures it cannot trace back to a lookup is not shown at all.
What do I need?
- Licence
- Enterprise
- Role
admin_user
Prerequisites
- The mcp-server and risk-analyst containers are running under the llm-analyst compose profile.
- An OpenAI-compatible model endpoint that emits tool_calls is reachable from the risk-analyst container — Ollama, vLLM or llama-server. Nothing on this page works without one.
- MCP_SHARED_TOKEN is set to the same value for both containers, and the cid_mcp_ro database role has been given the MCP_DB_PASSWORD password.
- The installed licence carries risk.llm_review, an Enterprise-tier key.
The analyst has no database connection string and no gateway credential. It reads through
cid-mcp-server, which logs in as a SELECT-only role, offers nineteen named tools and no SQL
tool at all. It writes nothing: your conversations are stored by the gateway, not by the analyst.
Note
This page and Risk analysis share one backend. The other page is the record of what the analyst decided about people on its own; this one is you asking it questions.
/assets/screenshots/analyst@0.4.pngWhat has to be running
The analyst and its MCP server are off by default — they are the only two services under the
llm-analyst compose profile:
mcp-server needs postgres healthy and answers 503 starting on /healthz until it has built
its dense search index, roughly 80 seconds on CPU for both reference corpora; its healthcheck
allows 180 seconds. risk-analyst waits for mcp-server to be healthy and allows 120 seconds of
its own. Compose reserves 1 GB for the MCP server and caps the analyst at 3 GB and 2 CPUs. Neither
downloads anything at boot: the embedding weights are baked into both images.
The model is not one of them. cid-risk-analyst speaks exactly one contract —
POST {LLM_BASE_URL}/chat/completions with OpenAI-schema tools — and the engine is yours to
choose:
The shipped default. LLM_BASE_URL points at http://host.docker.internal:11434/v1,
LLM_MODEL is qwen3:14b and LLM_ENGINE is ollama. Run Ollama natively — on a Mac a
containerised engine cannot reach the GPU:
Both settings are the server's to make: the OpenAI-compatible /v1 surface carries neither a
context length nor a keep-alive, so without them the weights unload after five minutes and the
engine may serve a shorter context than the analyst's prompts assume.
Warning
LLM_ENGINE selects the reasoning-suppression switch and nothing else. Setting it wrong fails
silently: the model thinks for hundreds of hidden tokens per round and the latency budget
disappears.
Ask a question
Open the page
Select AI Analyst at the top of the sidebar.
An empty conversation opens on Ask the analyst with four starter prompts, among them Which users tripped jailbreak detection this week? and Which users are locked right now, and why?
Type the question
Type into Ask about detections, users, filters… and send it.
The hint under the box states the contract: every answer is grounded in your own data and verified before it is shown. Ask in any language; the analyst answers in the language of the question.
Watch what it does
The status line reports the stage — Reading the question…, Read as: …, Thinking…, then Checking the answer….
A local 14B routinely takes over a minute to reach its first token under load. The stream has no deadline on purpose, and the analyst sends a heartbeat every 15 seconds so intermediaries do not cut it. Select Cancel to abandon the turn; the abort reaches the analyst and frees its worker slot.
Read the answer, then read the trail
Expand What it looked up (n) under the answer.
Each entry names the tool, its arguments, how many rows came back and how long it took. This is not a debugging aid: a question about "this week" that quietly ran over 30 days reads exactly like a correct answer until the parameters are visible.
Note
Rows come back as a real table, drawn from the tool result rather than typed by the model. That is deliberate — a model reciting a list as a sentence is where invented rows come from.
What it can and cannot look at
Every lookup goes through the MCP server's nineteen typed tools. There is no SQL tool, so the model cannot express a query the tools do not offer.
| It can read | It cannot read |
|---|---|
| Detections across every stream in the All Detections feed, and their counts | Any table not explicitly granted to cid_mcp_ro |
| Session transcripts, windowed around a detection and length-capped | ai_sessions.pii_map and the summarization history |
| Tenant profiles and group memberships | Account passwords, reset tokens and directory identifiers |
| Input and output filter configuration | Provider credentials |
| Past LLM reviews and the set-based risk analyses | ai_llm_reviews.callback_token_hash |
| The attack-knowledge corpus, and the MITRE ATLAS and OWASP reference releases | Your own earlier conversations with this page |
| Per-user activity summaries and aggregate statistics | Anything at all for writing — the role is SELECT-only and read-only at the transaction level |
Note
Your conversations with the analyst are deliberately not granted to the MCP role. They hold administrators' own questions, which are not evidence about a user — and granting them would let the analyst read its own operator's earlier phrasing back as a finding.
Scope follows your account. A superadmin's questions run fleet-wide; a group-scoped admin_user
gets the same tools pinned to the tenants they may read, and an identifier outside that set is
refused by name before anything is read.
When an answer is withheld
The analyst runs a deterministic verifier over its own answer before you see it, and the tokens are buffered until it passes — an answer that was streamed has already been shown.
| Check | Fails when |
|---|---|
| Figures trace | A number in the answer is in no tool result, and no arithmetic over two of them reaches it |
| Leaked vocabulary | A tool name, an envelope status or a hand-drawn table appears in the prose |
| Answer language | The answer is written in a different language from the question |
| False superlative | A comparison names a row that holds the extreme of no field. Off unless the deployment supplies the words |
| Invented labels | A quoted value appears in no result. Off by default |
A failure takes one correction round naming exactly what went wrong. A second failure quarantines the turn: you are told nothing could be confirmed, and the answer itself is discarded rather than shown with a caveat. The table survives quarantine when there was evidence, because its rows came from the tool result and the refusal never called those into question.
Danger
A quarantined turn never carries the withheld figure into the message that withholds it — that is why the refusal is written in a separate round that cannot see the answer, and why any refusal containing a digit is replaced by the neutral template The answer could not be completed. Ask again, narrowing the question.
Each finished answer is tagged Verified or Unverified and followed by its duration, token counts and round count. There is no price: the model is local, so a cost figure would be invented.
Manage conversations
Switch between conversations
Use the Conversations panel on the right, or Show conversations if you closed it.
Conversations are listed newest first with their relative age; the panel is remembered per browser. Your last open conversation reopens on your next visit.
Start a new one
Select New conversation.
The transcript is stored by the gateway, not the browser, so the model is sent the tool trail of earlier turns as well as their text — which is how it knows which lookups it already ran.
Delete one
Select the delete action on a conversation and confirm Delete conversation?
This will permanently delete this conversation and all its messages. This action cannot be undone. Its messages go with it.
Note
When the history stops fitting, the analyst folds the oldest turns into a recap inside the turn and draws Earlier conversation compacted · n turns folded across the transcript. The folded turns stay on screen; they are simply no longer what the model is reading. The Context window meter above the composer counts down to the next fold.
Fields reference
| Control | What it does |
|---|---|
| New conversation | Opens an empty conversation. The old one stays in the panel |
| Show conversations / Hide conversations | Toggles the right-hand panel. Remembered per browser |
| Search conversations | Filters the panel by title |
| Delete conversation | Removes a conversation and every message in it, permanently |
| Cancel | Aborts the turn in flight and frees the analyst's worker slot |
| What it looked up (n) | Expands the tool trail: tool, arguments, row count, duration |
| Context window meter | Prompt tokens this turn against the budget, and how far to the next compaction |
| Environment variable | Default | What it controls |
|---|---|---|
RISK_ANALYST_SERVICE_URL | http://risk-analyst:8019 | Where the gateway finds the analyst |
RISK_ANALYST_SERVICE_ENABLED | false | Whether automatic reviews are dispatched. It does not gate this page |
ANALYST_CHAT_STREAM_TIMEOUT | 0 (none) | A ceiling on one conversation turn. Not in docker-compose.yml; set it only to impose one |
LLM_BASE_URL | http://host.docker.internal:11434/v1 | The OpenAI-compatible model endpoint |
LLM_MODEL | qwen3:14b | The model name the endpoint serves |
LLM_ENGINE | ollama | Which reasoning-suppression switch to send |
ANALYST_CONTEXT_WINDOW | 32768 | What the prompt budgets assume. Startup reads what the engine really serves and lowers this to it |
MCP_SHARED_TOKEN | — | Bearer token between the analyst and the MCP server. Must match on both |
MCP_DB_PASSWORD | — | The cid_mcp_ro login password. The role cannot authenticate until it is applied |
Verify
curl -f http://localhost:8020/healthzanswers with a status that is notdegraded. Adegradedanswer with HTTP 503 means the database role lost its read-only settings.curl -f http://localhost:8019/healthreportsllm_reachableandmcp_connectedtrue, and a tool count above zero.- Ask Which input filters fired most often in the last 7 days? The answer arrives with a
drawn table, a Verified tag, and a tool trail naming
get_filter_configoraggregate_detection_statswith a non-zero row count. - Reload the page. The conversation is still there, its table redrawn and its meter restored — proof the turn was stored by the gateway rather than kept in the browser.
- This page writes no event log rows of its own. The analyst's automatic reviews do; see Risk analysis.
If it fails
- This feature (risk.llm_review) is not included in the installed license tier. Contact your CID
representative to upgrade. — a 403 with the code
FEATURE_NOT_LICENSED. This is an Enterprise key; on a lower tier the sidebar entry is hidden and the URL is refused. See Licence. - An error bubble reading connect ECONNREFUSED or Risk analyst is unavailable. — the
gateway could not reach the analyst. The profile is not up, or
RISK_ANALYST_SERVICE_URLis wrong. The turn is still stored, with the failure in place of the answer. - The answer is a refusal and the turn is tagged Unverified — the verifier could not trace the figures. Narrow the question, or name the window explicitly; the analyst withholding an answer is the rule working, not a fault.
- Every question fails after several prose rounds with nothing in the tool trail — the model
endpoint is not emitting
tool_calls. Test it directly before blaming the analyst: without--enable-auto-tool-choice --tool-call-parser hermeson vLLM, or--jinjaonllama-server,toolsis ignored entirely. - Answers are extremely slow and the trail shows long gaps — the engine is serving a smaller
context than
ANALYST_CONTEXT_WINDOW, or the weights unloaded between questions. The analyst logs an error at startup when the served context is short. mcp-servernever becomes healthy — thecid_mcp_rorole has no password. ApplyMCP_DB_PASSWORD, or set it by hand withALTER ROLE cid_mcp_ro PASSWORD '<secret>'.- A tool answers
not_supportednaming a stream or asearchcolumn — that grant has not been applied on this box. The call without the unsupported parameter still works. - This is a read-only demo account (viewer role) — actions and changes are disabled. — a
READ_ONLY_ROLErefusal. A viewer may open this page and read stored conversations, but asking a question is a POST and is refused.
Next steps
- Risk analysis — the automatic side of the same analyst: what it decided about repeated jailbreak attempts, and who it locked.
- All detections — the feed the analyst reads, with the filters you can hand it in a question.
- Licence — which tier carries
risk.llm_review.
Last updated on