Skip to main content
CID222 Docs

Ask the AI Analyst

Question your own detection data in plain language. The analyst runs on a local model, reads through read-only tools, and withholds any answer it cannot verify.

  • Version: 0.4
  • Feature: risk.llm_review
  • Role: admin_user
  • Type: task

AI Analyst is a chat over your own data. You ask a question in your own words; a local model looks the answer up through read-only tools, draws the rows it found as a table, and shows you which lookups it made. An answer whose figures it cannot trace back to a lookup is not shown at all.

What do I need?

Licence
Enterprise
Role
admin_user

Prerequisites

  • The mcp-server and risk-analyst containers are running under the llm-analyst compose profile.
  • An OpenAI-compatible model endpoint that emits tool_calls is reachable from the risk-analyst container — Ollama, vLLM or llama-server. Nothing on this page works without one.
  • MCP_SHARED_TOKEN is set to the same value for both containers, and the cid_mcp_ro database role has been given the MCP_DB_PASSWORD password.
  • The installed licence carries risk.llm_review, an Enterprise-tier key.

The analyst has no database connection string and no gateway credential. It reads through cid-mcp-server, which logs in as a SELECT-only role, offers nineteen named tools and no SQL tool at all. It writes nothing: your conversations are stored by the gateway, not by the analyst.

Note

This page and Risk analysis share one backend. The other page is the record of what the analyst decided about people on its own; this one is you asking it questions.

What has to be running

The analyst and its MCP server are off by default — they are the only two services under the llm-analyst compose profile:

docker compose --profile llm-analyst up -d mcp-server risk-analyst

mcp-server needs postgres healthy and answers 503 starting on /healthz until it has built its dense search index, roughly 80 seconds on CPU for both reference corpora; its healthcheck allows 180 seconds. risk-analyst waits for mcp-server to be healthy and allows 120 seconds of its own. Compose reserves 1 GB for the MCP server and caps the analyst at 3 GB and 2 CPUs. Neither downloads anything at boot: the embedding weights are baked into both images.

The model is not one of them. cid-risk-analyst speaks exactly one contract — POST {LLM_BASE_URL}/chat/completions with OpenAI-schema tools — and the engine is yours to choose:

The shipped default. LLM_BASE_URL points at http://host.docker.internal:11434/v1, LLM_MODEL is qwen3:14b and LLM_ENGINE is ollama. Run Ollama natively — on a Mac a containerised engine cannot reach the GPU:

ollama pull qwen3:14b
OLLAMA_CONTEXT_LENGTH=32768 OLLAMA_KEEP_ALIVE=-1 ollama serve

Both settings are the server's to make: the OpenAI-compatible /v1 surface carries neither a context length nor a keep-alive, so without them the weights unload after five minutes and the engine may serve a shorter context than the analyst's prompts assume.

Warning

LLM_ENGINE selects the reasoning-suppression switch and nothing else. Setting it wrong fails silently: the model thinks for hundreds of hidden tokens per round and the latency budget disappears.

Ask a question

Open the page

Select AI Analyst at the top of the sidebar.

An empty conversation opens on Ask the analyst with four starter prompts, among them Which users tripped jailbreak detection this week? and Which users are locked right now, and why?

Type the question

Type into Ask about detections, users, filters… and send it.

The hint under the box states the contract: every answer is grounded in your own data and verified before it is shown. Ask in any language; the analyst answers in the language of the question.

Watch what it does

The status line reports the stage — Reading the question…, Read as: …, Thinking…, then Checking the answer….

A local 14B routinely takes over a minute to reach its first token under load. The stream has no deadline on purpose, and the analyst sends a heartbeat every 15 seconds so intermediaries do not cut it. Select Cancel to abandon the turn; the abort reaches the analyst and frees its worker slot.

Read the answer, then read the trail

Expand What it looked up (n) under the answer.

Each entry names the tool, its arguments, how many rows came back and how long it took. This is not a debugging aid: a question about "this week" that quietly ran over 30 days reads exactly like a correct answer until the parameters are visible.

Note

Rows come back as a real table, drawn from the tool result rather than typed by the model. That is deliberate — a model reciting a list as a sentence is where invented rows come from.

What it can and cannot look at

Every lookup goes through the MCP server's nineteen typed tools. There is no SQL tool, so the model cannot express a query the tools do not offer.

It can readIt cannot read
Detections across every stream in the All Detections feed, and their countsAny table not explicitly granted to cid_mcp_ro
Session transcripts, windowed around a detection and length-cappedai_sessions.pii_map and the summarization history
Tenant profiles and group membershipsAccount passwords, reset tokens and directory identifiers
Input and output filter configurationProvider credentials
Past LLM reviews and the set-based risk analysesai_llm_reviews.callback_token_hash
The attack-knowledge corpus, and the MITRE ATLAS and OWASP reference releasesYour own earlier conversations with this page
Per-user activity summaries and aggregate statisticsAnything at all for writing — the role is SELECT-only and read-only at the transaction level

Note

Your conversations with the analyst are deliberately not granted to the MCP role. They hold administrators' own questions, which are not evidence about a user — and granting them would let the analyst read its own operator's earlier phrasing back as a finding.

Scope follows your account. A superadmin's questions run fleet-wide; a group-scoped admin_user gets the same tools pinned to the tenants they may read, and an identifier outside that set is refused by name before anything is read.

When an answer is withheld

The analyst runs a deterministic verifier over its own answer before you see it, and the tokens are buffered until it passes — an answer that was streamed has already been shown.

CheckFails when
Figures traceA number in the answer is in no tool result, and no arithmetic over two of them reaches it
Leaked vocabularyA tool name, an envelope status or a hand-drawn table appears in the prose
Answer languageThe answer is written in a different language from the question
False superlativeA comparison names a row that holds the extreme of no field. Off unless the deployment supplies the words
Invented labelsA quoted value appears in no result. Off by default

A failure takes one correction round naming exactly what went wrong. A second failure quarantines the turn: you are told nothing could be confirmed, and the answer itself is discarded rather than shown with a caveat. The table survives quarantine when there was evidence, because its rows came from the tool result and the refusal never called those into question.

Danger

A quarantined turn never carries the withheld figure into the message that withholds it — that is why the refusal is written in a separate round that cannot see the answer, and why any refusal containing a digit is replaced by the neutral template The answer could not be completed. Ask again, narrowing the question.

Each finished answer is tagged Verified or Unverified and followed by its duration, token counts and round count. There is no price: the model is local, so a cost figure would be invented.

Manage conversations

Switch between conversations

Use the Conversations panel on the right, or Show conversations if you closed it.

Conversations are listed newest first with their relative age; the panel is remembered per browser. Your last open conversation reopens on your next visit.

Start a new one

Select New conversation.

The transcript is stored by the gateway, not the browser, so the model is sent the tool trail of earlier turns as well as their text — which is how it knows which lookups it already ran.

Delete one

Select the delete action on a conversation and confirm Delete conversation?

This will permanently delete this conversation and all its messages. This action cannot be undone. Its messages go with it.

Note

When the history stops fitting, the analyst folds the oldest turns into a recap inside the turn and draws Earlier conversation compacted · n turns folded across the transcript. The folded turns stay on screen; they are simply no longer what the model is reading. The Context window meter above the composer counts down to the next fold.

Fields reference

ControlWhat it does
New conversationOpens an empty conversation. The old one stays in the panel
Show conversations / Hide conversationsToggles the right-hand panel. Remembered per browser
Search conversationsFilters the panel by title
Delete conversationRemoves a conversation and every message in it, permanently
CancelAborts the turn in flight and frees the analyst's worker slot
What it looked up (n)Expands the tool trail: tool, arguments, row count, duration
Context window meterPrompt tokens this turn against the budget, and how far to the next compaction
Environment variableDefaultWhat it controls
RISK_ANALYST_SERVICE_URLhttp://risk-analyst:8019Where the gateway finds the analyst
RISK_ANALYST_SERVICE_ENABLEDfalseWhether automatic reviews are dispatched. It does not gate this page
ANALYST_CHAT_STREAM_TIMEOUT0 (none)A ceiling on one conversation turn. Not in docker-compose.yml; set it only to impose one
LLM_BASE_URLhttp://host.docker.internal:11434/v1The OpenAI-compatible model endpoint
LLM_MODELqwen3:14bThe model name the endpoint serves
LLM_ENGINEollamaWhich reasoning-suppression switch to send
ANALYST_CONTEXT_WINDOW32768What the prompt budgets assume. Startup reads what the engine really serves and lowers this to it
MCP_SHARED_TOKENBearer token between the analyst and the MCP server. Must match on both
MCP_DB_PASSWORDThe cid_mcp_ro login password. The role cannot authenticate until it is applied

Verify

  1. curl -f http://localhost:8020/healthz answers with a status that is not degraded. A degraded answer with HTTP 503 means the database role lost its read-only settings.
  2. curl -f http://localhost:8019/health reports llm_reachable and mcp_connected true, and a tool count above zero.
  3. Ask Which input filters fired most often in the last 7 days? The answer arrives with a drawn table, a Verified tag, and a tool trail naming get_filter_config or aggregate_detection_stats with a non-zero row count.
  4. Reload the page. The conversation is still there, its table redrawn and its meter restored — proof the turn was stored by the gateway rather than kept in the browser.
  5. This page writes no event log rows of its own. The analyst's automatic reviews do; see Risk analysis.

If it fails

  • This feature (risk.llm_review) is not included in the installed license tier. Contact your CID representative to upgrade. — a 403 with the code FEATURE_NOT_LICENSED. This is an Enterprise key; on a lower tier the sidebar entry is hidden and the URL is refused. See Licence.
  • An error bubble reading connect ECONNREFUSED or Risk analyst is unavailable. — the gateway could not reach the analyst. The profile is not up, or RISK_ANALYST_SERVICE_URL is wrong. The turn is still stored, with the failure in place of the answer.
  • The answer is a refusal and the turn is tagged Unverified — the verifier could not trace the figures. Narrow the question, or name the window explicitly; the analyst withholding an answer is the rule working, not a fault.
  • Every question fails after several prose rounds with nothing in the tool trail — the model endpoint is not emitting tool_calls. Test it directly before blaming the analyst: without --enable-auto-tool-choice --tool-call-parser hermes on vLLM, or --jinja on llama-server, tools is ignored entirely.
  • Answers are extremely slow and the trail shows long gaps — the engine is serving a smaller context than ANALYST_CONTEXT_WINDOW, or the weights unloaded between questions. The analyst logs an error at startup when the served context is short.
  • mcp-server never becomes healthy — the cid_mcp_ro role has no password. Apply MCP_DB_PASSWORD, or set it by hand with ALTER ROLE cid_mcp_ro PASSWORD '<secret>'.
  • A tool answers not_supported naming a stream or a search column — that grant has not been applied on this box. The call without the unsupported parameter still works.
  • This is a read-only demo account (viewer role) — actions and changes are disabled. — a READ_ONLY_ROLE refusal. A viewer may open this page and read stored conversations, but asking a question is a POST and is refused.

Next steps

  • Risk analysis — the automatic side of the same analyst: what it decided about repeated jailbreak attempts, and who it locked.
  • All detections — the feed the analyst reads, with the filters you can hand it in a question.
  • Licence — which tier carries risk.llm_review.

Last updated on

On this page

Download PDF