Health matrix
Every service in the deployment: what it does, what stops working while it is down, the endpoint that reports its health, and the finding that fires.
- Version: 0.4
- Role: admin_user, viewer
- Type: reference
One row per service in the deployment. The What stops column is the one to read during an incident: several services fail in ways that leave the product serving, which is why a container that is down does not always match the symptom the user reported.
The full port, profile, memory-limit and image table is generated from the deployment definition: see Services, ports and profiles. This page adds what breaks and what to do.
How health is measured
The snapshot reports two different things, and they disagree usefully:
- Container health is the deployment's own healthcheck, in the
containerssection. It says whether the process is up. - Service health is the service's own endpoint, in the
servicessection. It says whether the service can do its job. A container can be running while its model failed to load.
Every service row is measured the same way — one request, a three-second cap, all in parallel — so the latencies are comparable with each other. A service that is not deployed is absent from the map rather than reported unhealthy.
Request path — the detection services
These run in the synchronous path. When one is down, the detection it performs does not happen; nothing retries it later.
| Service | What it does | What stops while it is down | Health endpoint | Finding |
|---|---|---|---|---|
nestjs-core | The gateway: chat, detection, admin API, ICAP listener | Everything. The dashboard loads and every call fails — and the snapshot itself cannot be collected, because the gateway serves it | GET /health | none: read the container's state and restart count instead |
ml-detector | Machine-learning PII detection over named entities | ML PII detection. Regex patterns still fire, so masking degrades rather than stopping | GET /api/v1/health | service-unhealthy-ml-detector |
hap-guard-v2 | Toxicity classification across 13 labels | Toxicity detection on input and output | GET /health | service-unhealthy-hap-guard-v2 |
attack-guard | Jailbreak and injection classification, plus SQL and script injection scanning | Jailbreak and injection detection | GET /api/v1/jailbreak/health | service-unhealthy-attack-guard |
language-detector | Language identification ahead of detection | Automatic language selection. Detection continues with the configured default | GET /health | service-unhealthy-language-detector |
Danger
A detector that is down does not fail closed. Traffic that would have been masked or blocked during the outage was not, and there is no retrospective pass — so after any detector outage, review what went through the gateway in that window.
Content services
| Service | What it does | What stops while it is down | Health endpoint | Finding |
|---|---|---|---|---|
ocr-service | Text extraction from images | Image analysis returns no text, so PII in screenshots is not seen | GET /api/v1/health | service-unhealthy-ocr-service |
document-parser | Text extraction from PDF and Office documents | Document analysis | GET /api/v1/health | service-unhealthy-document-parser |
redactor | Redaction of images and documents | Redacted downloads. Detection still reports what it found | GET /api/v1/health | service-unhealthy-redactor |
doc-classifier | Image document classification | The stronger image classifier. The image path falls back to text-based classification and machine-readable-zone detection | GET /health | service-unhealthy-doc-classifier |
hallucination-guard | Grounding checks on responses that carry context | Hallucination scoring. This runs behind the response, so users see no change | GET /api/v1/health | service-unhealthy-hallucination-guard |
report-renderer | Rendering of compliance reports to PDF and HTML | Report export. The data is still on screen | GET /health | service-unhealthy-report-renderer |
Cost and analysis services
| Service | What it does | What stops while it is down | Health endpoint | Finding |
|---|---|---|---|---|
prompt-compressor | Reduces input tokens before the provider call | Token reduction. Requests go to the provider uncompressed and cost more | GET /health | service-unhealthy-prompt-compressor |
cost-analyzer | Builds the model-routing policy from request history | Routing recommendations and projected savings | GET /health | service-unhealthy-cost-analyzer |
risk-analyst | The local-model analyst that reviews repeated detections | Analyst reviews. It reports healthy only when both the local model and the database tool server answer | GET /health | service-unhealthy-risk-analyst |
mcp-server | Read-only database access for the analyst | The analyst, which depends on it | GET /healthz | service-unhealthy-mcp-server |
llm-inference | Serves the local model the analyst uses | The analyst | Reported through the analyst's own health | service-unhealthy-risk-analyst |
deepteam | Red-team test execution | Filter testing runs | GET /health | service-unhealthy-deepteam |
code-safety | The external code auditor the gateway calls | Code Safety verdicts. Everything else keeps working | GET /api/v1/code/health | service-unhealthy-code-safety |
Platform
These have no application health endpoint of their own. Read them in the containers and
datastores sections instead.
| Component | What it does | What stops while it is down | Where to read it | Finding |
|---|---|---|---|---|
postgres | Every persisted record: tenants, credentials, filters, detections, sessions, events | Effectively everything. Lists come back empty rather than erroring | datastores.postgres | datastore-unreachable, migrations-pending |
redis | Rate-limit counters and the cluster lock | Rate limiting degrades open and scheduled work stops. Requests keep flowing, so it looks healthy | datastores.redis | datastore-unreachable |
caddy | Terminates 80 and 443 in front of the dashboard and the gateway | Everything, from the outside. Nothing answers on 443 | appliance.tls.servedBy | tls-443-not-served |
frontend | The dashboard | The dashboard. The API keeps serving its own clients | containers entry | container-restarting |
cid-proxy | Authentication and routing in front of the gateway | Whatever is routed through it, depending on the topology | containers entry | container-restarting |
reconfigure-agent | The host agent: host repair, detector re-tuning, and the host and container facts in the snapshot | Host repair, and the host and containers sections of the snapshot | host reads null | none — the snapshot stays silent on what it cannot measure |
Optional edge components
| Component | What it does | What stops while it is down | Where to read it | Finding |
|---|---|---|---|---|
cid-inline-proxy | The forward proxy that decrypts client traffic and calls the inspection listener | Inline inspection of browser traffic | containers entry | container-restarting |
cid-userid-agent | Attributes traffic to directory users | User attribution on inspected traffic; detections still record the address | GET /healthz | service-unhealthy-userid-agent — the snapshot keys this service as userid-agent |
ocr-service-gpu | The GPU build of the OCR service | Nothing extra — it replaces the CPU OCR service where a card is present | containers entry | container-oom |
Support tooling
These never run on a customer appliance. They exist for development and laboratory work, and a finding about one on a production box means the wrong profile was started.
| Component | What it does |
|---|---|
pgadmin | Database browser |
redis-commander | Redis browser |
mailpit | Captures outbound mail during testing |
openldap-test | A directory to test integration against |
weights-fetch | Fetches local-model weights at build time |
Reading a row during an incident
- Find the service in the
servicessection of the snapshot. If it is absent, it is not deployed — that is not a fault. - If
reachableis false, the problem is below the application: read the container's state and restart count. Follow service-unhealthy. - If
reachableis true andhealthyis false, read the error. A model that did not load is the usual answer, and on an air-gapped appliance the OCR service is the usual service — see ocr-models-missing-airgapped. - If the container has been killed for memory, the machine is the fault and not the service: container-oom.
Related
- Diagnostic snapshot — every key the snapshot carries.
- Guided runbooks — the checklist for each failure.
Last updated on