The full catalogue of symptom-driven runbooks, grouped by lifecycle phase and executable against the appliance's own diagnostics.

Each runbook starts from a symptom you can recognise, checks the appliance's own state, and names both the cause and the fix. Work one from the top: the checks are ordered so the cheapest and most likely cause is eliminated first.

Tip

Checks that read the diagnostics snapshot run themselves — in the appliance's own copy of this documentation you get a live verdict beside each one, with no model configured and no internet connection. Checks marked manual are the ones no tool can see; you answer those.

If a finding in the diagnostic snapshot sent you here, it named a runbook id: use the heading that carries it. If you are starting from a symptom in the install, first-boot, licensing, TLS or update phases, the install and activation hub is a shorter route.

Install

Provisioning stops at "6/8 container images"

The machine has no working path to the registry. Read the indented line above the failure: it separates a blocked port from a broken resolver, stale credentials and an inspected TLS connection.

Runbook install-provisioning-stops-at-images is not in this bundle. Run npm run docs:sync to pull the current corpus.

This network has no DHCP

The unattended boot entry expects an address to be offered. A segment that offers none needs the boot entry that asks for one, or a static address set from the appliance console.

Runbookinstall

This network has no DHCP, and the appliance comes up with no address

You might see: no IPv4 network (no default route) · the VM summary shows only an fe80:: address · the appliance has no IP address after the install · this network has no DHCP

Before you start

Is the virtual NIC attached to a network and connected at power-on, rather than left disconnected in the VM settings?expected · The adapter is connected and on the network you intend the appliance to serve.If not, go to /install/network-prerequisites first.
Have you confirmed this segment serves no DHCP, rather than serving it slowly or reserving by MAC?expected · No address is offered on this segment.If not, follow the install-provisioning-stops-at-images runbook instead.

Checks

1
Check which boot menu entry this appliance was installed from
expected · The install was started, or will be started, from the "set the network by hand — no DHCP" entry.
2
Check whether a static address was set from the appliance console instead
expected · An IPv4 address, gateway and name servers are set, and a default route exists.
3
Verify the name servers you configured actually answer
expected · Both queries answer. A configured resolver that times out is worse than none, because the failure looks like a network outage.

Causes and fixes

Reached when	Cause	Explanation, fix and verification
check 1 fails	`wrong-boot-entry` (warning)	The default boot entry is fully unattended and expects DHCP; it never asks for an address. On a segment with no DHCP that entry produces a machine with no network and an install that stops at the first pull. The dedicated entry shows a network page before it touches the disk. Fix: Reinstall from the boot entry "set the network by hand — no DHCP". The installer shows its own network page: select the interface, choose Edit IPv4 → Manual, fill in the subnet, address, gateway and name servers, then select Done. Nothing else is asked, and the address you type is the address the finished appliance keeps. Verify: expected · The appliance holds the address you typed, and provisioning ran to completion.
check 2 fails	`no-static-address` (critical)	No address was configured by hand and none was offered by the network, so the appliance has only a link-local address. Every outbound step — the registry pull, name resolution, the update channel — fails identically, which makes the cause hard to read from the symptoms. Fix: Set the address from the appliance console rather than reinstalling: press Alt+F1 for the console, then F2, and set the address, gateway and name servers. The console uses a US keyboard layout and warns when Caps Lock is on. Reboot afterwards — provisioning resumes by itself and keeps everything already downloaded. Verify: expected · The address you set is present and a default route exists.
check 3 fails	`resolver-unreachable` (critical)	A static configuration includes the name servers, and a wrong entry there produces an appliance that has an address and still resolves nothing. The installer then reports that it cannot resolve the registry with this network's DNS, with public resolvers, or from a pinned address. Fix: Correct the name servers on the console network page and reboot. If the site's resolvers cannot answer for the registry at all, add the address your CID222 contact gives you to /etc/hosts as a temporary bypass for the install window. Verify: expected · The query answers with the expected address.

The resource check fails before anything downloads

Memory and free disk are checked before the first pull. The installer's own thresholds predate the current supported minimum, so a run that only warned can still be undersized.

Runbookinstall

The installer stops on memory or free disk before anything downloads

You might see: this machine has 4096 MB of RAM — the appliance cannot run below 6 GB · only 12 GB free on / — the release images need about 40 GB · memory is 8192 MB; 16 GB is the designed size · the installer stopped at 0/8 this machine

Before you start

Do you know which package this appliance is licensed for? The RAM floor differs between packages.expected · You can name the package the appliance is being installed for.If not, go to /install/requirements first.

Checks

1
Read the memory the installer measured on this machine
expected · The reported memory is at least 32 GB, the supported minimum for the full stack.
2
Read the free disk the installer measured on the root filesystem
expected · At least 100 GB free, the supported minimum. The installer refuses below 20 GB and warns below 45 GB.
3
Check whether the kernel stopped the previous attempt
expected · No line reports "KILLED BY THE KERNEL".

Causes and fixes

Reached when	Cause	Explanation, fix and verification
check 1 fails	`ram-too-small` (critical)	The installer checks memory and free disk before it downloads anything, because an undersized machine otherwise fails an hour later as containers that will not stay up. It refuses below 6 GB and warns below 16 GB — but 16 GB was the previous design value, and the supported minimum for the full stack is now 32 GB. A run that produced only a warning can therefore still be a machine that will be OOM-killed under load. Fix: Power the machine off, raise the memory to at least 32 GB for the starter and professional packages or 64 GB for enterprise, and boot again. Images already downloaded are kept, so the retry is quick. See Sizing and requirements. Verify: expected · The step reports the raised memory and the run continues past it.
check 2 fails	`disk-too-small` (critical)	The release image set needs about 40 GB to download and unpack, and an update needs room for a second copy of every image on top of that. The installer refuses below 20 GB free and warns below 45 GB; the supported minimum for a production appliance is 100 GB. Fix: Give the machine a larger disk and reinstall. Retrying on the same disk cannot succeed — the installer's own message says so. Growing the virtual disk without growing the filesystem does not help either; the check reads free space on the root filesystem. Verify: expected · At least 100 GB free on the root filesystem.
check 3 fails	`oom-during-install` (critical)	The kernel killed the previous attempt for running out of memory. The installer keeps the reason and reports it on the next boot, because a machine that silently restarts its install looks like a network fault. Unpacking the image set is the memory peak of the whole install. Fix: Raise the memory to at least 32 GB and boot again. Everything already downloaded is kept. If the machine is already at 32 GB and is still killed, collect the console output and the log before changing anything else. Verify: expected · The run completes without a kernel kill. If that does not apply: Attach /var/log/cid-firstboot.log and the hypervisor's own memory configuration for the VM.

The appliance cannot get out

Name resolution, a TCP connection on 443 and an authenticated request through the site proxy fail independently. Each passes routinely while the others do not.

Runbookinstall

The appliance cannot get out — one or more egress tests failed

You might see: egress tests failed · the appliance cannot reach the registry · ETIMEDOUT after 5000ms · proxy returned 407 Proxy Authentication Required

Before you start

Is this appliance meant to reach the internet at all, rather than being deliberately air-gapped?diagnostics · appliance.channel = "offline"If not, go to /install/updates first.
Does this site require an outbound HTTP proxy for anything leaving the network?expected · You can say yes or no. A proxy-only network with no proxy configured produces a timeout, not a refusal, which is what makes this ambiguous.

Checks

1
Verify the appliance actually ran its outbound probes
diagnostics · appliance.egress
2
Read the name-resolution results for the hosts this appliance must reach
diagnostics · appliance.dns
3
Read whether an outbound proxy is configured on the appliance
diagnostics · appliance.proxy.configured = true
4
Read whether the release channel answered on the last attempt
diagnostics · appliance.update.channelReachable = true
5
Check whether a TLS-inspection device is presenting its own certificate to the appliance
expected · The issuer is a public certificate authority, not your organisation's inspection CA.

Causes and fixes

Reached when	Cause	Explanation, fix and verification
check 1 fails	`probes-disabled` (info)	The outbound probes are switched off for this deployment, so the snapshot carries no evidence either way and no egress finding can fire. This is a deliberate setting for a box that must make no outbound request at all. Fix: Re-enable the diagnostics egress probes on the gateway container and collect the snapshot again, or run the three tests by hand from the appliance host. The probes are the only outbound requests the appliance makes on its own. Verify: diagnostics · appliance.egress
check 2 fails	`dns-broken` (critical)	The appliance resolves with this network's resolver. A resolver that cannot answer for the provider or channel hosts fails every outbound call identically, and a browser on a laptop keeps working because it resolves over DNS-over-HTTPS. Nothing downstream of resolution can be diagnosed until this is fixed. Fix: Set a resolver that answers for the hosts in the DNS section of the snapshot, then collect the snapshot again. See Network prerequisites. Verify: diagnostics · appliance.dns
check 3 fails	`proxy-not-configured` (critical)	This site requires an outbound proxy and the appliance has none, so every outbound connection is attempted directly and times out. A timeout reads as a dead network rather than as a missing setting, which is why the network is reported as open. Fix: Configure the outbound proxy on the appliance and restart the gateway container so it re-reads the setting. Docker itself reads neither the shell environment nor apt's configuration, so a proxy set for the shell alone does not reach image pulls. Verify: diagnostics · appliance.proxy.configured = true
check 4 fails	`https-path-blocked` (critical)	Names resolve and the HTTPS path does not complete. DNS, a TCP connection on 443, and an authenticated HTTPS request fail independently: a firewall that permits ICMP and blocks 443 is ordinary, and Ubuntu's mirrors are served over port 80, so ping and apt both succeed while this fails. The three tests are not interchangeable. Fix: Allow direct outbound HTTPS from the appliance to the hosts the snapshot names as failing, or route them through the site proxy and configure it on the appliance. Read each failing target's detail line in the snapshot before changing a rule — a 407 is a proxy credential problem, a timeout is a blocked path, and a TLS error is inspection. Verify: diagnostics · appliance.update.channelReachable = true
check 5 fails	`tls-inspection-untrusted` (critical)	A middlebox terminates TLS and presents a certificate the appliance does not trust, so the connection fails at the handshake. The detail line reports a TLS error rather than a timeout, which is the only way to tell this apart from a blocked port. Fix: Install your site's CA certificate on the appliance so it trusts the inspection device, or exempt the appliance's outbound destinations from inspection. Both are legitimate; pick the one your security team already operates. Verify: expected · The failing egress targets report ok, or the issuer is now trusted by the appliance.

The machine has less RAM than the package needs

The full stack was measured at about 30 GiB in use. Below the package minimum the kernel evicts whichever detection service is largest when pressure peaks.

Runbookinstall

The machine has less RAM than the licensed package needs

You might see: This box has 16.0 GB of RAM; the package needs 32 GB · ML services will be OOM-killed under load · memory is 8192 MB; 16 GB is the designed size · the appliance runs but services keep being evicted

Before you start

Is the host agent reporting? Without it the snapshot cannot see the machine's memory at all.diagnostics · host.memTotalMb = 0If not, go to /troubleshoot/diagnostic-snapshot first.
Which package is this appliance licensed for? The minimum differs between them.diagnostics · appliance.license.tierIf not, go to /install/licensing first.

Checks

1
Compare total memory against the 32 GB minimum for the full stack
diagnostics · host.memTotalMb = 32768
2
For a deployment that processes documents and images, compare against the 64 GB recommendation
diagnostics · host.memTotalMb = 65536
3
Check whether the undersizing has already cost a service
expected · No container has been killed for memory.

Causes and fixes

Reached when	Cause	Explanation, fix and verification
check 1 fails	`below-minimum` (critical)	The full stack was measured at about 30 GiB in use. Below 32 GB the machine cannot hold the detection services and their models at the same time, so the kernel evicts whichever is largest when pressure peaks. The product works in a demo and fails under the first real load, which is why this is worth fixing before go-live rather than after. Fix: Raise the machine's memory to at least 32 GB and reboot. If the hardware cannot be changed, run only the core gateway and the three mandatory detection services and switch the rest off deliberately — a 16 GB box is a development configuration, not a supported production one. See Sizing and requirements. Verify: diagnostics · host.memTotalMb = 32768
check 2 fails	`below-recommendation` (warning)	32 GB carries the stack; documents, images and OCR add the largest transient allocations in the product, and 64 GB is what keeps them from competing with the detectors. A GPU is recommended for that work and never required. Fix: Raise the machine to 64 GB before enabling document and image analysis for a whole organisation, or keep those features to a pilot group until you can. Verify: diagnostics · host.memTotalMb = 65536
check 3 fails	`already-losing-services` (critical)	The undersizing is no longer theoretical: a container has been killed or is restarting. Any detection that should have run while a detector was down did not run, and nothing retries it. Fix: Raise memory first, then review the detections for the affected window — a detector that was down did not fail closed. See A container was killed for running out of memory. Verify: expected · No container reports an out-of-memory kill and restart counts are stable.

A filesystem is nearly full

An update needs room for a second copy of every image, and the state directory fails silently when the root filesystem fills.

Runbookinstall

A filesystem is nearly full

You might see: is 94% full (6 GB free of 100 GB) · an update needs room for a second copy of every image · no space left on device · the update fails part way through

Before you start

Is the host agent reporting? The gateway container sees its own overlay filesystem, not the machine's disks.diagnostics · host.disksIf not, go to /troubleshoot/diagnostic-snapshot first.

Checks

1
Read the free space on the root filesystem
diagnostics · host.disks[/].freeGb = 20
2
Read the free space where Docker keeps its images and volumes
diagnostics · host.disks[/var/lib/docker].freeGb = 40
3
Read how large the CID database has grown
diagnostics · datastores.postgres.dbSizeMb = 50000

Causes and fixes

Reached when	Cause	Explanation, fix and verification
check 1 fails	`root-filling` (critical)	The appliance writes its state, its logs and the update spool to the root filesystem. When it fills, writes that the product wraps in a log-and-continue handler fail silently — the setup state, the licence and update intents all behave as if they were saved and are not. Fix: Free space or grow the disk, then restart the gateway container so it re-reads its state. Check the journal and any log files the site collects: an unrotated log is the usual reason an appliance that ran for months fills up. Verify: diagnostics · host.disks[/].freeGb = 20
check 2 fails	`image-store-filling` (critical)	An update downloads a second copy of every image before it switches over, so an appliance needs roughly the size of the release set free at the moment it updates. Without that room the update fails part way through and rolls back, which reads as a broken update rather than a full disk. Fix: Remove images the appliance no longer runs, then update. Do not delete volumes to make room: they hold the database, the detections and the appliance identity. Verify: diagnostics · host.disks[/var/lib/docker].freeGb = 40
check 3 fails	`retention-too-long` (warning)	The detection and event history grows with use, and nothing prunes it by default. On a busy appliance the database becomes the largest single consumer of the disk, and the first symptom is an update that will not fit rather than a database that complains. Fix: Reduce the retention window for detections and event logs to what your compliance obligation actually requires, and confirm the size falls afterwards. Export first if the records must be kept outside the appliance. Verify: diagnostics · datastores.postgres.dbSizeMb = 50000

The host clock is not synchronised

Licence expiry, certificate validity, token lifetimes and detection timestamps all come from this clock. A drifting one produces three failures that each look like something else.

Runbookinstall

The host clock is not synchronised

You might see: The host clock is not synchronised · clock is not synchronised — if the registry login fails on a certificate error, this is why · a valid licence is reported as expired · tokens are rejected as expired immediately after sign-in

Before you start

Is the host agent reporting? The clock fact comes from the machine, not from the container.diagnostics · host.kernelIf not, go to /troubleshoot/diagnostic-snapshot first.

Checks

1
Read whether the host clock is disciplined by a time service
diagnostics · host.timeSynced = true
2
Verify the appliance can reach a time source at all
expected · The status reports the clock as synchronised, or names a server the network permits.
3
Check whether the skew has already aged the licence out
diagnostics · appliance.license.state = "active|trial"

Causes and fixes

Reached when	Cause	Explanation, fix and verification
check 1 fails	`no-time-source` (warning)	Licence expiry, certificate validity, session-token lifetimes and every detection timestamp are decided by this clock. A drifting clock therefore expires licences early, rejects tokens that were just issued, and makes a current certificate look not-yet-valid — three failures that each look like something else. During an install it also breaks the registry login with a certificate error that reads like a network fault. Fix: Enable time synchronisation on the appliance host and point it at a time source this network permits. Restart the gateway container afterwards so anything it cached at boot is re-evaluated against the corrected clock. Verify: diagnostics · host.timeSynced = true
check 2 fails	`ntp-blocked` (warning)	A time client that cannot reach a server leaves the clock free-running, and the failure is silent: nothing on the appliance reports that time is wrong until something dated fails. Many sites block outbound time traffic and provide an internal server instead. Fix: Point the appliance at the site's own time server, or open the time protocol to a permitted external one. See Network prerequisites. Verify: expected · The clock reports as synchronised.
check 3 fails	`licence-mis-aged` (critical)	The skew has already pushed the licence out of its validity window, so the product blocks exactly as it would for a real expiry. Requesting a renewal at this point produces a licence that will look expired too. Fix: Correct the clock first, then re-read the licence state. Request a renewal only if the state is still expired once time is right. See The licence has expired. Verify: diagnostics · appliance.license.state = "active\|trial"

Sample accounts with published passwords still exist

Appliance images ship with sample seeding off. Where it was on, the demo tenants are working sign-ins for anyone who has read the documentation.

Runbookinstall

Sample accounts with published passwords exist on this appliance

You might see: Sample accounts are seeded · sarah_smith exists on this box · demo users are present on a customer appliance · the documented demo password works

Before you start

Is this appliance serving real users, rather than a demonstration or training instance where the sample tenants are the point?diagnostics · appliance.deploymentMode = "development"If not, go to /develop/getting-started/introduction first.

Checks

1
Read whether sample-account seeding is switched off for this deployment
diagnostics · posture.seedSampleAccounts = false
2
Check whether the sample tenants still exist, since switching seeding off does not delete what was already created
expected · No sample tenant is present.

Causes and fixes

Reached when	Cause	Explanation, fix and verification
check 1 fails	`sample-seeding-enabled` (info)	The seeder inserts the provider catalogue and, unless sample accounts are switched off, demo tenants whose passwords are published in the documentation. Appliance images already ship with sample accounts disabled, so this is normal on a developer box and a finding anywhere else — a seeded demo user is a working sign-in for anyone who has read the docs. Fix: Set `SEED_SAMPLE_ACCOUNTS` to `false` in the deployment's environment file and restart the gateway container, so a later reseed does not recreate them. Do not disable seeding altogether: that also skips the provider catalogue and leaves the product unable to resolve a model. Verify: diagnostics · posture.seedSampleAccounts = false
check 2 fails	`sample-accounts-still-present` (warning)	Switching the setting off stops the next seed from creating the accounts; it does not remove accounts that already exist. A box that was demonstrated before it was handed over keeps them, with their published passwords, until someone deletes them. Fix: Delete the sample tenants under Tenants, or change their passwords if their sessions and detections must be preserved for a demonstration. Then confirm the administrator account you actually use is not one of the seeded ones. Verify: expected · The sign-in is refused.

The OCR service has no models on an air-gapped appliance

An image built before the model check shipped can carry the engine and not its models: healthy on a networked machine, which fetches them on demand, and fatal air-gapped.

Runbookinstall

The OCR service has no models on an air-gapped appliance

You might see: HTTP 503: det/rec models not found · ocr-service answered but is not healthy · image analysis returns no text on the appliance and works on the dev box · OCR works when the machine has internet and fails when it does not

Before you start

Does this deployment use image or document analysis? The OCR service is opt-in for many customers.expected · At least one of them is in use. If neither is, an unhealthy OCR service costs nothing today.If not, go to /troubleshoot/health-matrix first.
Does the OCR service answer at all, rather than being absent or down?diagnostics · services[ocr-service].reachable = trueIf not, follow the service-unhealthy runbook instead.

Checks

1
Read whether the OCR service reports itself healthy
diagnostics · services[ocr-service].healthy = true
2
Rule out an out-of-memory kill, which produces an unhealthy OCR service for a different reason
expected · oomKilled is false.

Causes and fixes

Reached when	Cause	Explanation, fix and verification
check 1 fails	`models-absent-from-image` (warning)	OCR models are baked into the image at build time. Constructing the engine downloads only the orientation classifier on current versions — detection and recognition arrive on first use — so a build that stopped at construction produced an enormous image holding a few megabytes of models. That image is healthy on a networked machine, which fetches the rest on demand, and fatal on an air-gapped one, which cannot. The build now forces a real prediction and refuses the image when detection and recognition are absent, so this affects images built before that check. Fix: Replace the OCR image with one from a build that passes the model check; there is nothing to repair inside a running container, because the files were never in it. When rebuilding, pass a cache-busting build argument — the pre-download layer is otherwise served from cache and a failed download looks permanent. Verify: diagnostics · services[ocr-service].healthy = true If that does not apply: Attach the support bundle and the image digest the snapshot reports for ocr-service.
check 2 fails	`ocr-oom` (critical)	The OCR service holds the largest transient allocations in the product, so on an undersized machine it is the first thing the kernel kills. The result — an unhealthy OCR service and image analysis that returns nothing — looks identical to a missing model until you read the kill flag. Fix: Follow A container was killed for running out of memory. 64 GB is the recommendation for a deployment that processes documents and images. Verify: expected · oomKilled is false and the service reports healthy.

A first-boot provisioning step fails on the console

Provisioning aborts on the steps the appliance cannot work without and warns on the ones it can, so a warned step leaves a usable appliance with something missing and nothing further happening on its own.

Runbookinstall

A first-boot provisioning step fails on the console

You might see: docker install failed · docker daemon will not start · secret generation failed · console UI install FAILED

Before you start

Did first boot get past the container-image step, or is it stuck there?expected · The failure is at a step other than the container-image pull.If not, follow the install-provisioning-stops-at-images runbook instead.

Checks

1
Read the first-boot log rather than the console
expected · The log names the step and carries the failing command's output.
2
Separate a step that aborted provisioning from one that warned and continued
expected · You can say whether provisioning aborted.
3
Check whether Docker installed and is running
expected · The daemon answers.
4
Check that this box generated its own per-unit secrets
diagnostics · posture.jwtSecretPlaceholder = false
5
For the HTTPS step, check whether the front-end container exists yet
diagnostics · appliance.tls.servedBy

Causes and fixes

Reached when	Cause	Explanation, fix and verification
check 1 fails	`no-log` (info)	The console scrolls and the operator's screen can freeze at an early step while later lines go to a console nobody is watching. The log file is the authoritative record of the run. Fix: Read `/var/log/cid-firstboot.log`. Where the run was resumed, it records where the previous attempt stopped and why. Verify: expected · The failing step is identified.
check 2 fails	`aborted-run` (critical)	Provisioning aborts on the steps the appliance cannot work without and warns on the ones it can. A warned step leaves a usable appliance with something missing — the console UI, or trust for your site's own CA — and no further action happens on its own. Fix: Fix what the failing step names, then re-run provisioning. For the console UI specifically, the message gives the command to re-run it by hand. Verify: expected · Every step reports ok.
check 3 fails	`docker-missing` (critical)	The ISO carries Docker as baked packages and only falls back to the internet when they are absent. On an air-gapped install that fallback cannot work, so a missing package set is fatal rather than slow. Fix: Use the ISO built for this deployment — the air-gap ISO carries everything — and re-install. A network install needs outbound HTTPS from the appliance. Verify: expected · The daemon answers.
check 4 fails	`secrets-not-generated` (critical)	Every appliance generates its own database password, token signing secret, proxy key and PII cipher key at first boot. That is the property a golden image cannot have, so a failure here leaves the unit either unbootable or sharing secrets, and provisioning aborts. Fix: Re-run provisioning so the secrets step completes. Where the appliance is already running with a placeholder secret, follow The token signing secret is a placeholder. Verify: diagnostics · posture.jwtSecretPlaceholder = false
check 5 fails	`https-before-stack` (warning)	The HTTPS helper configures the front end that the stack creates. Run before the stack has started, it finds no container and says so; run after, it succeeds. The appliance is HTTPS-only, so until it completes there is nothing to browse to. Fix: Let the stack start, then run the HTTPS helper the console names. See The install finished but the dashboard does not load. Verify: diagnostics · appliance.tls.servedBy If that does not apply: Attach `/var/log/cid-firstboot.log` and the console transcript. The diagnostics snapshot does not exist yet at this stage.

First boot and the setup wizard

The install finished but the dashboard does not load

An ISO-installed appliance is HTTPS-only, and the front end refuses to start with a certificate it cannot use. A blank page, a refused connection and a brief upstream error have three causes.

Runbookfirst-boot

The install finished but the dashboard does not load

You might see: the browser shows a blank white page · https://<ip>/ refuses the connection · HTTPS sidecar FAILED · 3 of 16 containers are running; these are not:

Before you start

Did the console reach the end of the run, rather than stopping with a red FAILED line?expected · The run reached "8/8 starting the stack" and printed a status summary.If not, follow the install-provisioning-stops-at-images runbook instead.
Are you opening https://, not http://? An ISO-installed appliance is HTTPS-only.expected · The URL begins with https://. A plain http:// page renders blank, because the frontend sends an upgrade-insecure-requests policy.If not, go to /install/setup-wizard first.
Have you given the stack a few minutes since the machine booted? The ML services load their models on first start.expected · At least five minutes have passed since boot.

Checks

1
Read the console's own count of running containers
expected · Every container is running, and no line reports "N of M containers are running".
2
Verify something holds port 443 on the appliance host
diagnostics · appliance.tls.servedBy
3
Verify the gateway itself is reachable
diagnostics · services[nestjs-core].reachable = true
4
Verify the gateway container is not restarting repeatedly
diagnostics · containers[nestjs-core].restarts = 3

Causes and fixes

Reached when	Cause	Explanation, fix and verification
check 2 fails	`https-listener-not-installed` (critical)	The HTTPS front end is installed by a separate step at the end of the run, and it is not optional: the dashboard sends an upgrade-insecure-requests policy, so a plain http:// page renders blank and https:// is refused outright when the listener never installed. The console reports this at the end of the run. Fix: Re-run the HTTPS installer on the appliance host with `sudo bash /opt/cid-install/install-https.sh`, then reload the dashboard over https://. The appliance serves a self-signed certificate at this point, which is correct — it has no name and no CA yet, and the site replaces the certificate later. Verify: diagnostics · appliance.tls.servedBy
check 1 fails	`containers-not-up` (critical)	The install itself succeeded and one or more services did not stay up. The console names each one and prints the last three lines of its log, which is where the reason is. An exit code of 137 is a kernel out-of-memory kill and means the machine is undersized, not that the service is broken. Fix: Read the last lines the console printed for each container that is down. For exit 137, raise the machine's memory to at least 32 GB and reboot — see The machine is below the package's RAM minimum. For anything else, read the full log with `docker compose -f /opt/cid/docker-compose.release.yml logs`. Verify: expected · The summary reports the full container count running and names none as down.
check 3 fails	`gateway-down` (critical)	The dashboard is a static frontend that calls the gateway for everything. When the gateway does not answer, the page can load and then fail every request, or the front end can return 502 while it waits for an upstream that never arrives. Fix: Read the gateway container's log from the most recent start and fix the boot error it reports. A placeholder JWT secret refuses boot by design, and pending migrations stop the readers of the new columns — both surface here as a gateway that will not serve. Verify: diagnostics · services[nestjs-core].reachable = true If that does not apply: Attach the support bundle, which carries the gateway log tail.
check 4 fails	`gateway-crashloop` (critical)	The gateway is restarting, so requests that land mid-restart fail with errors unrelated to what the reader was doing. A sustained restart count is the signature of a boot failure, not of load. Fix: Follow A container keeps restarting, which separates an out-of-memory kill from a configuration refusal. Verify: diagnostics · containers[nestjs-core].restarts = 3

The connectivity step returns 500

Almost always a permissions problem on the state directory, not a network problem — the connectivity probe itself may well have succeeded before the write failed.

Runbookfirst-boot

The setup wizard's connectivity step returns 500

You might see: POST /setup/connectivity → 500 · setup wizard connectivity test fails with a server error · Internal server error on the network step of the wizard · wizard will not advance past connectivity

Before you start

Is the appliance still in setup mode, with the wizard unfinished?diagnostics · appliance.setupComplete = falseIf not, go to /administer/settings first.
Does the gateway answer at all — does the wizard page itself load?diagnostics · services[nestjs-core].reachable = trueIf not, follow the update-version-zero-or-500 runbook instead.

Checks

1
Verify the gateway container is running and is not restarting repeatedly
diagnostics · containers[nestjs-core].restarts = 3
2
Verify the gateway can write its state directory, where the wizard persists each step
expected · The command exits 0 and prints nothing. A permission-denied error confirms this cause.
3
Verify the hostnames the connectivity step probes actually resolve
diagnostics · appliance.dns
4
Read the outbound connectivity probe results
diagnostics · appliance.egress

Causes and fixes

Reached when	Cause	Explanation, fix and verification
check 2 fails	`state-dir-permissions` (critical)	The wizard persists the result of each step under /var/lib/cid. When that directory is owned by root but the gateway process runs unprivileged — the state after a manual `mkdir` or a restore that did not preserve ownership — the write throws, and the handler surfaces it as a 500 rather than as a permissions error. The connectivity probe itself may well have succeeded. Fix: Correct the ownership of the state directory on the host so the gateway's user can write it, then restart the gateway container and retry the step. Do not make the directory world-writable; it holds appliance identity material. Verify: expected · The probe exits 0 and the wizard advances to the next step.
check 1 fails	`gateway-crashloop` (critical)	The gateway container is restarting, so a request that lands mid-restart fails with a 500 that has nothing to do with connectivity. The wizard is reporting the symptom of a boot failure — most often a refused placeholder JWT secret, or a migration that did not apply. Fix: Read the gateway container logs from the start of the most recent restart and fix the boot error itself. Check the security posture first: a placeholder `JWT_SECRET` refuses boot by design, and pending migrations mean the schema readers will fail. Verify: diagnostics · containers[nestjs-core].restarts = 3 If that does not apply: Attach the support bundle, which carries the gateway log tail.
check 3 fails	`no-dns` (warning)	The appliance cannot resolve the hostnames it is asked to probe. A resolver that times out rather than answering NXDOMAIN makes the step hang and then fail, which reads as a server error. Fix: Set a reachable DNS server in the appliance network configuration and re-run the step. On a DHCP-less install the resolver must be configured explicitly — see Network prerequisites. Verify: diagnostics · appliance.dns
check 4 fails	`egress-blocked` (warning)	Name resolution works but nothing gets out. DNS, TCP/TLS to a provider, and an authenticated request through the configured proxy fail independently — a successful DNS lookup is not evidence that the other two work, and treating them as interchangeable is what makes this step confusing. Fix: Read the three probe results separately in the diagnostics snapshot and fix the one that actually failed. If the site requires an outbound proxy, configure it on the appliance before retrying; an unset proxy on a proxy-only network produces a timeout, not a refusal. Verify: diagnostics · appliance.egress

The wizard finishes and the appliance asks for setup again

The completion write is wrapped in a handler that logs and continues, so an unwritable directory produces a wizard that completes and a product that locks itself again at the next restart.

Runbookfirst-boot

The setup wizard finishes and the appliance asks for setup again

You might see: 423 SETUP_REQUIRED · Appliance setup is not complete. Finish first-boot setup to use the product. · the wizard completes and the dashboard sends me back to it · every page returns 423 after setup

Before you start

Did the wizard reach its final step and report that setup was complete, rather than stopping on an earlier step?expected · The wizard reaches the final step. If it stops earlier, the failing step is the problem, not the lock.If not, follow the setup-connectivity-500 runbook instead.
Can you still sign in as the administrator the wizard created?expected · Sign-in succeeds. Authentication stays reachable while the product is locked, by design.If not, go to /install/setup-wizard first.

Checks

1
Read whether the gateway considers setup complete
diagnostics · appliance.setupComplete = true
2
Verify the gateway can write the state directory the wizard persists into
expected · The command exits 0 and prints nothing.
3
Verify the gateway is not restarting between wizard steps
diagnostics · containers[nestjs-core].restarts = 3

Causes and fixes

Reached when	Cause	Explanation, fix and verification
check 2 fails	`state-dir-not-writable` (critical)	The wizard persists its state to a file under the shared state directory. That write is wrapped in a handler that logs and continues, so an unwritable directory produces a wizard that completes in memory, unlocks the product until the container restarts, and then locks it again with the state file still absent. The usual causes are a host directory created by hand as root, a restore that did not preserve ownership, or a read-only bind mount. Fix: Correct the ownership of the state directory on the host so the gateway's own user can write it, and confirm the mount is not read-only. Then restart the gateway container and run the wizard once more. Do not make the directory world-writable; it holds appliance identity material. Verify: expected · The probe exits 0 and the dashboard loads the product rather than the wizard.
check 1 fails	`completion-not-persisted` (critical)	The gateway reports setup as incomplete, so its setup guard answers 423 on every route outside the small allowlist the wizard, authentication and health need. Everything else in the product is locked until that state changes, which is why the dashboard sends you back. Fix: Establish why the completion was not recorded — an unwritable state directory is the common cause — then complete the wizard again. The guard is inert the moment the state records completion; no restart is needed to unlock, only to reload a lost state. Verify: diagnostics · appliance.setupComplete = true
check 3 fails	`restart-loses-memory-state` (critical)	The gateway is restarting during the wizard. Each restart reloads the state from disk, so progress that only ever existed in memory is lost and the wizard begins again from the step the file last recorded. Fix: Fix the restart first, then run the wizard once, uninterrupted. See A container keeps restarting. Verify: diagnostics · containers[nestjs-core].restarts = 3

Settings the appliance saves do not survive a restart

The same root cause seen from the other side: setup state, the licence, update intents and the diagnostics hand-off share one directory and fail silently together.

Runbookfirst-boot

Settings the appliance saves do not survive a restart

You might see: the setup state directory is not writable · setup-state write failed · the licence I uploaded is gone after a restart · the wizard completes and comes back

Before you start

Is the gateway container running, so a write could be attempted at all?diagnostics · services[nestjs-core].reachable = trueIf not, follow the first-boot-no-dashboard runbook instead.

Checks

1
Write and remove a probe file in the shared state directory as the gateway's own user
expected · The command exits 0 and prints nothing. Permission denied or a read-only filesystem confirms this runbook.
2
Repeat the probe in each subdirectory the product writes
expected · All four exit 0.
3
Check that the filesystem holding the state directory has space
diagnostics · host.disks[/].freeGb = 2

Causes and fixes

Reached when	Cause	Explanation, fix and verification
check 1 fails	`ownership-or-readonly-mount` (critical)	The gateway runs unprivileged and writes its durable state under one shared directory: the setup state file, the licence, update intents and the diagnostics hand-off all live there. Every one of those writes is wrapped in a handler that logs and continues, so an unwritable directory produces a product that appears to accept changes and forgets them at the next restart rather than one that reports an error. Fix: Correct the ownership of the state directory on the host so the gateway's user can write it, and confirm the bind mount is not read-only. Restart the gateway container afterwards so it re-reads whatever state exists. Do not make the directory world-writable — it holds appliance identity material. Verify: expected · The probe exits 0 and the setting survives the restart.
check 2 fails	`subdirectory-owned-by-root` (critical)	The parent directory is writable and one subdirectory is not — the state after a manual mkdir as root, or a restore that recreated the tree without preserving ownership. Only the feature that writes into that one subdirectory fails, which is why the symptom looks specific to licensing or to updates rather than to storage. Fix: Correct the ownership of every subdirectory under the state root, not just the root itself, then restart the gateway container. Verify: expected · All four exit 0.
check 3 fails	`disk-full` (critical)	The filesystem holding the state directory has no room, so writes fail for a reason that has nothing to do with permissions. The product behaves identically in both cases because both are swallowed by the same handler. Fix: Free space on the appliance, or grow the disk. See A filesystem is nearly full. Verify: diagnostics · host.disks[/].freeGb = 2

The providers step shows an empty list

Provider and model rows come from the database seeder, not from code. Nothing is broken; the data was never inserted.

Runbookfirst-boot

The setup wizard's providers step shows an empty list

You might see: GET /setup/providers returns an empty list · no providers to choose from in the wizard · provider dropdown is empty during setup · cannot select OpenAI or Anthropic in the wizard

Before you start

Is the appliance still in setup mode?diagnostics · appliance.setupComplete = falseIf not, go to /administer/credentials first.

Checks

1
Verify the gateway can reach PostgreSQL — the provider list is read from it, not hard-coded
diagnostics · datastores.postgres.reachable = true
2
Verify no migrations are pending, so the provider and model tables exist in their current shape
diagnostics · datastores.postgres.migrationsPending
3
Ask the setup API for the provider list
GET /setup/providers

Causes and fixes

Reached when	Cause	Explanation, fix and verification
check 3 fails	`seed-never-ran` (critical)	The provider and model rows come from the database seeder, not from code. When seeding on boot is disabled the tables stay empty, and the wizard faithfully shows nothing to choose from. Nothing is broken — the data was never inserted. Fix: Enable seeding on boot, or run the seeder once against this database, then reload the wizard step. The seed inserts the provider rows and their models; it does not insert credentials, so you still supply your own API key in the next step. Verify: GET /setup/providers If that does not apply: If the seeder runs without error and the list is still empty, attach the support bundle and the seeder log.
check 1 fails	`database-unreachable` (critical)	The gateway cannot reach PostgreSQL, so every database-backed list is empty rather than erroring visibly. The providers step is simply the first place in the wizard where that becomes obvious. Fix: Start the database container and wait for its healthcheck to pass, then reload the step. If the container is running but unreachable, check that the gateway's database host, port and credentials match what the database is actually serving. Verify: diagnostics · datastores.postgres.reachable = true
check 2 fails	`migrations-pending` (critical)	The compiled migration chain contains migrations that are not recorded in the schema, so the provider or model tables are missing columns the current code selects. The query fails or returns nothing, and the wizard shows an empty list. Fix: Apply the pending migrations and restart the gateway. The entrypoint applies them on boot whenever synchronise is off, so a non-empty pending list on a booted appliance means that run failed — read the gateway log from the boot that failed rather than re-running blindly. Verify: diagnostics · datastores.postgres.migrationsPending

The token signing secret is a placeholder

A published placeholder lets anyone with the source mint an administrator token. An unset secret is different and not safe either: every restart invalidates every session.

Runbookfirst-boot

The JWT signing secret is a placeholder or unset

You might see: JWT_SECRET is a known placeholder · the gateway refuses to boot with a message about the signing secret · everyone is signed out after every restart · tokens stop working when the container restarts

Before you start

Is this box used by anyone other than you, or does it hold real tenant data?diagnostics · appliance.deploymentMode = "development"If not, go to /develop/getting-started/authentication first.

Checks

1
Read whether the effective signing secret is a placeholder or unset
diagnostics · posture.jwtSecretPlaceholder = false
2
Verify the gateway actually started, since a known placeholder refuses boot by design
diagnostics · services[nestjs-core].reachable = true

Causes and fixes

Reached when	Cause	Explanation, fix and verification
check 1 fails	`placeholder-secret` (critical)	Every session token this appliance issues is signed with this secret, and it also derives the key that protects stored directory bind passwords. A published placeholder means anyone with the source can mint an administrator token for this appliance. An unset secret is different but not safe: the gateway generates an ephemeral one per boot, so every restart invalidates every session. Fix: Generate 64 hex characters with `openssl rand -hex 64`, set `JWT_SECRET` to that value in the appliance's environment file, and restart the gateway container. Rotating the secret signs everyone out once, which is the intended cost. Re-enter any directory bind password afterwards: it was encrypted with a key derived from the old secret. Verify: diagnostics · posture.jwtSecretPlaceholder = false
check 2 fails	`boot-refused` (critical)	Startup refuses a known placeholder rather than serving with it, so the gateway does not come up at all and every page reports the front end's own upstream error. The refusal is deliberate — a product signing tokens with a public secret is worse than one that will not start. Fix: Set a real `JWT_SECRET` as above and start the container again. Read the container's first log lines to confirm the refusal was about the secret rather than about something else that also fails at boot. Verify: diagnostics · services[nestjs-core].reachable = true

A wizard step is refused

Some refusals are the wizard protecting a running appliance — it will not re-provision one whose setup is already complete — and some are a step that cannot be completed as configured.

Runbookfirst-boot

A first-boot wizard step is refused

You might see: setup already complete · an account already exists · cannot complete: required steps are missing · default tenant-group 'All Users' not found

Before you start

Does the wizard load, rather than the appliance answering 423 everywhere?expected · The wizard renders.If not, follow the setup-wizard-423-loop runbook instead.

Checks

1
Check whether setup has in fact already been completed
diagnostics · appliance.setupComplete = true
2
For a refusal to finish, read which steps are still outstanding
expected · Every critical step is done.
3
For the provider step, check that the default tenant group exists
expected · The group exists under **Tenant Groups**.
4
For the connectivity step, check the proxy fields together
expected · Either the toggle is off, or a URL is present.
5
Check whether a previous host-side change is still being applied
expected · No reconcile is pending.

Causes and fixes

Reached when	Cause	Explanation, fix and verification
no check points here	`already-complete` (info)	Once setup is complete the wizard's write endpoints refuse, so a bookmarked wizard URL or a stale browser tab produces this rather than re-running provisioning against a live appliance. Fix: Close the wizard and use the dashboard. Change anything it set from Settings instead. Verify: expected · It loads and you are signed in.
check 2 fails	`critical-step-outstanding` (warning)	Completion unlocks the product, so it requires the steps the product cannot run without. The refusal does not say which one, which is why the status list is the thing to read. Fix: Finish the outstanding critical steps, then press Finish again. Verify: diagnostics · appliance.setupComplete = true
check 3 fails	`default-group-missing` (warning)	The wizard attaches the first provider credential to a named default group rather than to the administrator personally, so that everyone can use it. With no such group the step is refused rather than creating a credential nobody can resolve. Fix: Create the group named by `DEFAULT_TENANT_GROUP` (default All Users), or set that variable to a group that exists, then run the step again. An unseeded database is the usual cause — see The providers step shows an empty list. Verify: expected · It completes.
check 4 fails	`proxy-toggle-without-url` (info)	Accepting the toggle with no URL would write an empty proxy configuration and report success, shipping a box with no egress and no sign of why. The step fails loudly instead. Fix: Enter the proxy URL, or turn the toggle off if this network has direct egress. Verify: expected · It completes.
check 5 fails	`reconcile-in-flight` (info)	Steps that change the host — TLS mode in particular — are applied by a host-side reconcile, and only one may be in flight. A second request is refused rather than queued behind it. Fix: Wait for the pending reconcile to finish, then repeat the step. Verify: expected · It completes. If that does not apply: Attach the support bundle. Where the state directory is at fault, several wizard steps fail together — see Settings the appliance saves do not survive a restart.

Wrong credentials, an inactive account, an account the directory owns and a directory that could not be reached are four deliberately distinguishable refusals, because each needs a different person to act.

Runbookaccess

A user cannot sign in to the dashboard

You might see: Invalid credentials · Account is inactive · Account not found · User not found

Before you start

Does the dashboard load its sign-in page at all, rather than showing a setup or licence lock?expected · A sign-in form. A setup lock (423) or a licence block (402) is a different problem.If not, follow the setup-wizard-423-loop runbook instead.

Checks

1
Establish whether this account is local or comes from the directory
expected · You can say which of the two it is.
2
For a directory account, check that the directory answered
expected · The directory test succeeds.
3
Read whether the account is active
expected · The account is active.
4
For a user who signs in and is immediately signed out again, check the token signing secret
diagnostics · posture.jwtSecretPlaceholder = false

Causes and fixes

Reached when	Cause	Explanation, fix and verification
check 1 fails	`wrong-authority` (info)	Local accounts and directory accounts fail in different places. A directory account's password lives in Active Directory; the gateway never stores it and cannot change it, so a password reset attempted here is refused rather than silently ignored. Fix: Reset a directory account's password in the directory. Reset a local account's password from Tenants, which produces a reset link delivered by the configured mechanism. Verify: expected · Sign-in succeeds.
check 2 fails	`directory-unreachable` (critical)	When the directory cannot be reached the gateway says so explicitly rather than reporting a wrong password, because the two need opposite responses. Local accounts keep working throughout — content policy and enforcement never consult the directory. Fix: Follow The directory cannot be reached or bound. Verify: expected · It succeeds, and the user signs in.
check 3 fails	`account-inactive` (warning)	An inactive account is refused at sign-in with a distinct message. Deactivation is how an account is retired without deleting the detections and sessions attributed to it. Fix: Re-activate the account under Tenants, or point the user at the account they should be using. Verify: expected · Sign-in succeeds.
check 4 fails	`jwt-secret-unstable` (critical)	Tokens are signed with `JWT_SECRET`. Unset, the gateway derives an ephemeral per-boot secret, so every restart invalidates every session; a known placeholder value is worse still, because anyone with the source can mint an administrator token. Fix: Follow The token signing secret is a placeholder. Verify: diagnostics · posture.jwtSecretPlaceholder = false If that does not apply: Attach the support bundle. Include whether the user is local or directory-managed and the exact message shown — the four refusals above are deliberately distinguishable.

The product refuses an action with 402, 403 or 423

Licence state, role, read-only demo accounts and incomplete setup all refuse here, and each carries a machine-readable code. The error codes page reads them one by one.

Runbookaccess

The product refuses an action with 402, 403 or 423

You might see: READ_ONLY_ROLE · This is a read-only demo account (viewer role) — actions and changes are disabled. · ROLE_NOT_FOR_CHAT · FEATURE_NOT_LICENSED

Checks

1
Read the machine-readable code in the error body, not the sentence
expected · You can name the code.
2
For a 423, check whether first-boot setup ever completed
diagnostics · appliance.setupComplete = true
3
For a 402, read the licence state
diagnostics · appliance.license.state = "active|trial"
4
For FEATURE_NOT_LICENSED, read which tier is installed and whether the feature is in it
diagnostics · appliance.license.tier
5
For a 403 naming a role, read what the account's role may do
expected · The role holds the page or capability the action needs.

Causes and fixes

Reached when	Cause	Explanation, fix and verification
check 1 fails	`no-code-in-body` (info)	A refusal with no code did not come from the licence guard, the setup guard or the role layer. Something else produced it — most often the provider, or a validation error on the request body. Fix: Read the HTTP status and the response body, and find the request in Event Logs. Verify: expected · You can attribute the refusal.
check 2 fails	`setup-not-complete` (critical)	Until first-boot setup completes, every product route answers 423 with `SETUP_REQUIRED`. Administration stays reachable so the wizard can be finished without a rescue procedure. Fix: Follow The wizard finishes and the appliance asks for setup again. Verify: diagnostics · appliance.setupComplete = true
check 3 fails	`licence-not-valid` (critical)	With enforcement on and no valid licence, product routes answer 402 `LICENSE_EXPIRED` while administration stays available. An expired licence and a licence that never verified produce the same status and different fixes. Fix: Follow The licence has expired and the product is locked, and check the host clock before requesting a renewal. Verify: diagnostics · appliance.license.state = "active\|trial"
check 4 fails	`feature-not-in-tier` (warning)	Licence tiers gate whole modules. The guard refuses the endpoint and the dashboard hides or disables the page, so the symptom is a feature that visibly exists and never works rather than one that is absent. Fix: Compare the installed tier against the generated tier table and ask your CID representative to add the feature. See Licensing. Verify: expected · It is accepted.
check 5 fails	`role-lacks-access` (info)	Role refusals are deliberate and are not licence problems. The `viewer` role in particular is read-only by design: it passes role checks on GET, HEAD and OPTIONS and is refused globally on everything that mutates, including downloading signed licence material. The capability model is computed in shadow mode and enforces nothing yet — what enforces today is the page matrix and the fixed roles. Fix: Give the account a role that holds the access, or perform the action as an administrator. See Access control. Verify: expected · It is accepted. If that does not apply: Include the code, the HTTP status, the route and the account's role.

Licensing

Two failures dominate activation, and they share one root cause: an appliance that demands a licence it is structurally unable to verify.

License upload fails with 400 "rejected: signature"

The licence file is usually fine. The appliance has no signing public key to check it against, so every licence looks forged.

Runbooklicensing

License upload fails with 400 "rejected: signature"

You might see: 400 rejected: signature · 400 rejected: ENOENT · License upload → 400 · licence file will not upload

Before you start

Has the first-boot setup wizard been completed on this appliance?diagnostics · appliance.setupComplete = trueIf not, follow the setup-connectivity-500 runbook instead.
Is the file you are uploading the licence issued for THIS appliance's installation id, unmodified since it was issued?expected · The two ids are identical, character for character.If not, go to /install/licensing first.

Checks

1
Verify the licence trust anchor exists on the host
diagnostics · appliance.license.trustAnchorPresent = true
2
Verify the appliance reports an installation id
diagnostics · appliance.license.installationId
3
Verify the host clock is disciplined by NTP
diagnostics · host.timeSynced = true
4
Read the licence state the gateway reports
GET /admin/license/status → {"state":"active|trial"}

Causes and fixes

Reached when	Cause	Explanation, fix and verification
check 1 fails	`missing-anchor` (critical)	The licence-signing public key (/etc/cid/license-pubkey.pem) was never baked into this image, or was removed. Verification has nothing to check the signature against, so every licence — including a perfectly valid one — is rejected as forged. The API surfaces this as a 400 with a signature or ENOENT message rather than as a missing-key error, which is why the licence file gets blamed. Fix: Restore the trust anchor on the host and restart the gateway container, then upload the licence again. On an appliance the anchor is part of the image: an image that lacks it was built wrong and the fix is to re-run the host repair, not to hand-place a key of unknown provenance. Follow Licensing and activation; if the appliance was built from an image that shipped without the anchor, collect the support bundle and escalate rather than sourcing a key yourself. Verify: diagnostics · appliance.license.trustAnchorPresent = true If that does not apply: Attach the support bundle. The image build, not the appliance, is at fault.
check 2 fails	`null-installation-id` (critical)	The appliance reports no installation id, so no licence can have been issued for it and no licence can bind to it. This normally means the identity was never generated during first boot, or the file holding it is not readable by the gateway container. Fix: Re-run the first-boot identity step and confirm the id appears on Settings → License. Request a licence only once a stable id is shown; a licence issued against a null id is unusable. See Licensing and activation. Verify: diagnostics · appliance.license.installationId
check 3 fails	`clock-skew` (warning)	Licence validity is bounded by dates. A host clock that is wrong by more than the licence's grace window makes a valid licence appear not-yet-valid or expired, and the error surfaces at the same place as a signature failure. Fix: Correct the host clock and enable NTP or chrony, then restart the gateway container and upload the licence again. See Network prerequisites for the NTP egress the appliance needs. Verify: diagnostics · host.timeSynced = true
check 4 fails	`wrong-licence-for-this-box` (critical)	The trust anchor, installation id and clock are all correct, so the signature check is meaningful — and it fails. The licence was issued for a different installation id, was modified in transit (a mail client re-wrapping lines is the usual culprit), or was truncated. Fix: Re-download the licence file from the issuing portal, transfer it as a binary attachment or over a checksum-verified copy, and upload it again without opening it in an editor. If the installation id in the licence does not match this appliance, request a re-issue. Verify: GET /admin/license/status → {"state":"active\|trial"} If that does not apply: Attach the support bundle and the installation id shown on Settings → License.

The appliance has no licence trust anchor

The same missing key before anyone uploads anything. With enforcement off the appliance quietly runs its trial, so the gap surfaces only when the trial ends.

Runbooklicensing

This appliance has no licence trust anchor and can only run the trial

You might see: No license trust anchor (/etc/cid/license-pubkey.pem) · no licence can be verified on this box · the appliance only ever runs the built-in trial · every licence file is rejected, whichever one we upload

Before you start

Is this an appliance built from CID222 install media, rather than a developer stack started from the compose file?diagnostics · appliance.deploymentMode = "appliance"If not, go to /develop/getting-started/introduction first.

Checks

1
Read whether the licence-signing public key exists on the host
diagnostics · appliance.license.trustAnchorPresent = true
2
Read whether licence enforcement is switched on, which decides how bad the missing anchor is
diagnostics · appliance.license.requireLicense = false

Causes and fixes

Reached when	Cause	Explanation, fix and verification
check 1 fails	`anchor-absent` (warning)	The licence-signing public key is part of the appliance image and lives at /etc/cid/license-pubkey.pem. Verification has nothing to check a signature against without it, so every licence — including a valid one — is rejected as forged. With enforcement off the appliance still runs its built-in trial, which is why the missing anchor can go unnoticed until the trial ends. Fix: Run the host repair from Settings → System Updates, which re-establishes the files the image should carry, then re-check the snapshot. An image that still lacks the anchor after repair was built wrong: collect the support bundle and escalate rather than placing a key of unknown provenance by hand. Verify: diagnostics · appliance.license.trustAnchorPresent = true If that does not apply: Attach the support bundle. The image build, not the appliance, is at fault.
check 2 fails	`enforcement-on-without-anchor` (critical)	Enforcement demands a valid licence and the appliance cannot verify one, so every licensed route answers 402 no matter what is uploaded. The appliance is enforcing a check it is structurally unable to pass. Fix: Restore the anchor as above and upload the licence issued for this installation id. See License upload fails with 400 signature. Turning enforcement off is a lab measure only — it disables the entitlement checks the tier relies on. Verify: GET /admin/license/status → {"state":"active\|trial"}

Every request returns 402 after the wizard completes

Licence enforcement is on and no licence resolves as valid. Administration stays reachable so you can fix it without a rescue procedure.

Runbooklicensing

Every request returns 402 LICENSE_EXPIRED after the wizard completes

You might see: 402 LICENSE_EXPIRED · chat returns 402 after finishing setup · the dashboard loads but every action fails with payment required · product blocked immediately after setup

Before you start

Did the setup wizard complete successfully?diagnostics · appliance.setupComplete = trueIf not, follow the setup-connectivity-500 runbook instead.
Do the dashboard and login work, with only the product routes (chat, detection, inspection, image and document analysis) returning 402?expected · The first page loads and the second returns 402. If both fail, this is not a licensing problem.If not, go to /troubleshoot/health-matrix first.

Checks

1
Read whether licence enforcement is switched on for this deployment
diagnostics · appliance.license.requireLicense = true
2
Read the licence state the gateway resolved at boot
diagnostics · appliance.license.state = "active|trial"
3
Verify the licence trust anchor exists, since without it no licence can ever resolve as active
diagnostics · appliance.license.trustAnchorPresent = true
4
Check how long the installed licence has left
diagnostics · appliance.license.expiresAt

Causes and fixes

Reached when	Cause	Explanation, fix and verification
check 3 fails	`enforcement-on-without-anchor` (critical)	CID_REQUIRE_LICENSE is true and the licence-signing public key is absent. The guard demands a valid licence, and verification cannot succeed without the anchor, so every licensed route answers 402 no matter what is uploaded. The appliance is enforcing a check it cannot pass. Fix: Restore the trust anchor and upload the licence, following License upload fails with 400 signature. Turning enforcement off is a deliberate, temporary measure for a lab only — never the fix on a customer appliance, because it disables the entitlement checks the tier relies on. Verify: diagnostics · appliance.license.trustAnchorPresent = true
check 2 fails	`no-valid-license` (critical)	Enforcement is on and the licence resolved to a state other than active or trial — missing, invalid or expired. The setup wizard does not install a licence, so a freshly completed appliance with enforcement on reaches exactly this state until one is uploaded. Fix: Upload the licence issued for this appliance under Settings → License. If no licence has been issued yet, request one against the installation id shown on that page. See Licensing and activation. Verify: GET /admin/license/status → {"state":"active\|trial"}
check 4 fails	`license-genuinely-expired` (critical)	A licence is installed and verifies, but its validity period has ended. Enforcement then blocks the product routes while leaving administration reachable, so the appliance can be re-licensed without a rescue procedure. Fix: Upload a renewed licence. If the expiry looks wrong, check the host clock first — a skewed clock ages a valid licence out early and produces exactly this state. Verify: diagnostics · appliance.license.state = "active\|trial"
check 1 fails	`unexpected-402-without-enforcement` (warning)	Enforcement is off, so the licence guard should not be blocking anything. A 402 in this state means the value the gateway read at boot differs from the value now configured — usually the container was started before the setting changed and has not been restarted. Fix: Restart the gateway container so it re-reads its configuration, then retry. If 402 persists with enforcement off, collect the support bundle: the effective configuration and the configured one disagree. Verify: diagnostics · appliance.license.requireLicense = false If that does not apply: Attach the support bundle, including the effective posture section.

The licence has expired and the product is locked

An appliance that worked and now blocks its product routes. Check the clock before requesting a renewal.

Runbooklicensing

The licence has expired and licensed endpoints answer 402

You might see: License expired / product locked · 402 LICENSE_EXPIRED · License state is "expired" · the trial ran out

Before you start

Can you still sign in and open the dashboard, with only the product routes failing?expected · The first loads and the second returns 402. The licence path stays reachable while the product is locked, by design.If not, go to /troubleshoot/health-matrix first.

Checks

1
Read the licence state the gateway resolved
diagnostics · appliance.license.state = "active|trial"
2
Verify the host clock is disciplined, because a skewed clock ages a valid licence out early
diagnostics · host.timeSynced = true
3
Verify the trust anchor exists, since without it no renewal can ever verify either
diagnostics · appliance.license.trustAnchorPresent = true
4
Read the licence status the API reports
GET /admin/license/status → {"state":"active|trial"}

Causes and fixes

Reached when	Cause	Explanation, fix and verification
check 1 or 4 fails	`expired-or-invalid` (critical)	A licence is installed and its validity period has ended, or it no longer verifies. The guard then blocks the licensed routes — chat, detection, inspection, image and document analysis — with 402 while leaving administration reachable, so the appliance can be re-licensed without a rescue procedure. Fix: Upload a renewed licence under Settings → License. Request the renewal against the installation id shown on that page: a licence issued for a different id will not verify here. See Licensing and activation. Verify: diagnostics · appliance.license.state = "active\|trial"
check 2 fails	`clock-skew-fake-expiry` (warning)	Licence validity is bounded by dates read from the host clock. A clock that is wrong by more than the grace window makes a current licence look expired, and the product blocks exactly as it would for a real expiry. Fix: Correct the host clock and enable time synchronisation, then restart the gateway container and re-read the licence state before requesting a renewal. See The host clock is not synchronised. Verify: diagnostics · host.timeSynced = true
check 3 fails	`renewal-cannot-verify` (critical)	The trust anchor is absent, so the renewal you are about to upload will be rejected for the same reason the current licence no longer verifies. Uploading first wastes the maintenance window. Fix: Restore the trust anchor before uploading anything. See This appliance has no licence trust anchor. Verify: diagnostics · appliance.license.trustAnchorPresent = true

TLS and certificates

The certificate has expired, or nothing is serving 443

The umbrella runbook: a refused connection and an expired certificate look different to the user and share a cause more often than you would expect.

Runbooktls

The certificate has expired, or nothing is serving 443

You might see: ERR_CERT_DATE_INVALID · NET::ERR_CERT_AUTHORITY_INVALID · connection refused on 443 · the dashboard cannot be reached over HTTPS

Before you start

Are you reaching the appliance by the name the certificate was issued for, rather than by IP address?expected · The name you typed appears on the certificate.If not, go to /install/naming-dns-tls first.
Does the appliance respond on the network — does it answer a ping or an HTTP request on another port?expected · At least one of the two connects. If neither does, this is a network or power problem, not a TLS one.If not, go to /install/network-prerequisites first.

Checks

1
Verify something is actually holding port 443 on the appliance host
diagnostics · appliance.tls.servedBy
2
Verify the appliance has a certificate to serve
diagnostics · appliance.tls.mode = "none"
3
Read how many days the served certificate has left
diagnostics · appliance.tls.daysLeft = 0
4
Check whether the certificate is inside the renewal window — 30 days, the `CID_TLS_RENEW_DAYS` default `appliance/tls/ensure-tls.sh` renews an appliance-issued leaf at and warns about a customer certificate at
diagnostics · appliance.tls.daysLeft = 30
5
Verify the appliance root CA is present, so clients can be made to trust an appliance-issued certificate
diagnostics · appliance.tls.caPresent = true

Causes and fixes

Reached when	Cause	Explanation, fix and verification
check 1 fails	`nothing-on-443` (critical)	No process holds 443, so the client's connection is refused before any TLS handshake begins. The browser reports a connection error rather than a certificate error, which is why this does not look like a TLS problem at first. It follows a failed certificate rollout: the front-end refuses to start with an unusable certificate rather than serving a broken one. Fix: Run the host repair from Settings → System Updates, which re-establishes the HTTPS front end and re-applies the appliance's certificate intent. Then reload the dashboard. If repair reports a certificate problem, resolve that first — the front end will keep refusing to start until the certificate it is told to serve is valid. Verify: diagnostics · appliance.tls.servedBy If that does not apply: Attach the support bundle, which carries the host HTTPS inventory.
check 2 fails	`no-certificate` (critical)	The appliance has no certificate configured at all — neither one issued by its own CA nor an uploaded one. Nothing can be served on 443 until it does. Fix: Issue a certificate from the appliance CA under Settings → Certificates, or upload one your organisation's CA signed against a CSR generated there. See Naming, DNS and TLS. Verify: diagnostics · appliance.tls.mode = "none"
check 3 fails	`certificate-expired` (critical)	The served certificate is past its expiry. Browsers refuse the connection outright, and clients that were configured to trust the appliance — the inline proxy's own callers, the browser extension, the endpoint agent — fail closed rather than degrading. Fix: Issue a replacement now. An appliance-CA certificate is reissued in place from Settings → Certificates; an externally signed one needs a fresh CSR, signature and upload. Check the host clock before blaming the certificate: a skewed clock makes a valid certificate look expired. Verify: diagnostics · appliance.tls.daysLeft = 0
check 4 fails	`certificate-expiring-soon` (warning)	The certificate is still valid but inside the renewal window. Renewing now avoids an outage that would otherwise arrive without warning, and gives time for an external CA to turn a CSR around. Fix: Renew the certificate before it expires, following Naming, DNS and TLS. If the certificate is externally signed, start now — the signing turnaround, not the appliance, is the long pole. Verify: diagnostics · appliance.tls.daysLeft = 30
check 5 fails	`ca-missing` (warning)	The appliance root CA is absent, so clients cannot be made to trust an appliance-issued certificate and the public root-CA download has nothing to serve. Every client then reports an untrusted-authority error even when the certificate itself is current. Fix: Re-create the appliance CA under Settings → Certificates, redistribute the root certificate from the appliance's public download path, and reissue the server certificate. Note that reissuing invalidates trust that clients had already established, so plan the redistribution before you reissue. Verify: diagnostics · appliance.tls.caPresent = true

Nothing is listening on 443

Connections are refused before any handshake, so the browser reports a network error rather than a certificate error.

Runbooktls

Nothing is listening on 443

You might see: Nothing is listening on 443 · connection refused on 443 · https://<ip>/ refuses the connection · the dashboard cannot be reached over HTTPS

Before you start

Does the appliance answer on the network at all?expected · At least one connects. If neither does, this is a network or power problem rather than a TLS one.If not, go to /install/network-prerequisites first.

Checks

1
Read what holds port 443 on the appliance host
diagnostics · appliance.tls.servedBy
2
Verify the appliance has a certificate to serve at all
diagnostics · appliance.tls.mode = "none"
3
Verify the dashboard container the listener proxies to is running
diagnostics · containers[frontend].state = "running"

Causes and fixes

Reached when	Cause	Explanation, fix and verification
check 1 fails	`listener-absent` (critical)	No process holds 443, so the client's connection is refused before any handshake begins and the browser reports a connection error rather than a certificate error. On a freshly installed appliance the usual reason is that the HTTPS front end never installed; on a running one it follows a failed certificate rollout, because the front end refuses to start with an unusable certificate rather than serving a broken one. Fix: On a fresh install, re-run the HTTPS installer on the host with `sudo bash /opt/cid-install/install-https.sh`. On a running appliance, run the host repair from Settings → System Updates, which re-establishes the front end and re-applies the certificate intent. If repair reports a certificate problem, fix that first — the front end will keep refusing to start until the certificate it is told to serve is valid. Verify: diagnostics · appliance.tls.servedBy If that does not apply: Attach the support bundle, which carries the host HTTPS inventory.
check 2 fails	`no-certificate-to-serve` (critical)	The appliance has no certificate configured — neither one issued by its own CA nor an uploaded one — so there is nothing for a listener to present and nothing can be served on 443. Fix: Issue a certificate from the appliance CA under Settings → Certificates, or upload one your organisation's CA signed against a signing request generated there. See Naming, DNS and TLS. Verify: diagnostics · appliance.tls.mode = "none"
check 3 fails	`frontend-down` (critical)	The listener terminates TLS and proxies to the dashboard container. With that container down, 443 either refuses connections or answers with an upstream error, depending on which half failed first. Fix: Start the dashboard container and read its log if it exits again. See A container keeps restarting. Verify: diagnostics · containers[frontend].state = "running"

The certificate is expiring or has expired

Clients configured to trust this appliance fail closed rather than degrading, so enforcement stops at the moment the dashboard becomes unreachable.

Runbooktls

The appliance HTTPS certificate is expiring or has expired

You might see: The appliance HTTPS certificate expires in 6 day(s) · ERR_CERT_DATE_INVALID · certificate expired · your connection is not private

Before you start

Is the host clock correct? A skewed clock makes a current certificate look expired.diagnostics · host.timeSynced = trueIf not, follow the time-not-synced runbook instead.

Checks

1
Read how many days the served certificate has left
diagnostics · appliance.tls.daysLeft = 0
2
Check whether the certificate has entered the renewal window — 30 days, the `CID_TLS_RENEW_DAYS` default in `appliance/tls/ensure-tls.sh`
diagnostics · appliance.tls.daysLeft = 30
3
Read which kind of certificate the appliance serves, because the renewal path differs
diagnostics · appliance.tls.mode = "unknown"

Causes and fixes

Reached when	Cause	Explanation, fix and verification
check 1 fails	`already-expired` (critical)	The served certificate is past its expiry. Browsers refuse the connection, and the clients that were configured to trust this appliance — the browser extension, the endpoint agent, the inline proxy's callers — fail closed rather than degrading, so enforcement stops at the same moment the dashboard becomes unreachable. Fix: Issue a replacement now. A certificate issued by the appliance's own CA is reissued in place from Settings → Certificates; an externally signed one needs a fresh signing request, a signature and an upload. See Naming, DNS and TLS. Verify: diagnostics · appliance.tls.daysLeft = 0
check 2 fails	`inside-renewal-window` (warning)	The certificate is still valid and inside the window where renewal should start. An externally signed certificate needs the signing turnaround as well as the appliance work, and that turnaround, not the appliance, is the long pole. Fix: Renew now rather than at expiry. Generate the signing request from Settings → Certificates, have it signed, and upload the result. An appliance-CA certificate is reissued in place with no external dependency. Verify: diagnostics · appliance.tls.daysLeft = 30
check 3 fails	`mode-unknown` (warning)	The appliance cannot say which certificate it serves, so neither renewal path can be recommended and the expiry reading may be about a certificate that is no longer the one on 443. This is what the snapshot reports when the host TLS inventory is missing. Fix: Run the host repair from Settings → System Updates so the TLS inventory is written again, then re-read the snapshot before renewing anything. Verify: diagnostics · appliance.tls.mode = "unknown"

The appliance refuses the certificate you uploaded

Certificate, key and chain are validated in sequence and each has its own vocabulary. Fixing the wrong file leaves the same message in place.

Runbooktls

The appliance refuses the certificate you uploaded

You might see: cert is not a PEM X.509 certificate · cert is a CA certificate; upload the server (leaf) certificate, with the CA in chain · the private key does not match the certificate · key is not a PEM private key (encrypted keys are not accepted — decrypt it first)

Before you start

Is the appliance serving HTTPS at all today, even with a certificate you want to replace?diagnostics · appliance.tls.servedByIf not, follow the tls-443-not-served runbook instead.

Checks

1
Read which of the three files the refusal is about — the certificate, the key or the chain
expected · You can say which file is being refused.
2
Confirm you uploaded the server certificate, not the CA
expected · CA:FALSE on the certificate field; the CA belongs in the chain field.
3
Confirm the key matches the certificate
expected · The moduli match and the key is not passphrase-protected.
4
Confirm the certificate carries a subjectAltName
expected · The hostname the appliance is reached by is listed as a SAN.
5
Confirm the chain is ordered leaf-side first
expected · Each link issues the one before it.
6
Check whether a previous change is still being applied on the host
expected · No reconcile is pending.

Causes and fixes

Reached when	Cause	Explanation, fix and verification
check 1 fails	`cannot-tell-which-file` (info)	Certificate, key and chain are validated in sequence and each has its own vocabulary. Fixing the wrong file leaves the same message in place and looks like the upload is broken. Fix: Read the refusal literally; it names the file it is about before it names the defect. Verify: expected · Either it is accepted, or the message names a different file.
check 2 fails	`uploaded-the-ca` (info)	A CA certificate in the certificate field is refused rather than installed, because serving a CA as a leaf produces a site every browser rejects. The same check runs the other way for the outbound-inspection CA field, which refuses a leaf. Fix: Put the server certificate in the certificate field and the issuing CA in the chain field. Verify: expected · The upload is accepted.
check 3 fails	`key-mismatch` (warning)	A key that does not pair with the certificate cannot complete a handshake, so the appliance refuses it at upload instead of after the restart that would take the dashboard offline. Encrypted keys are refused for the same reason: nothing can supply the passphrase at boot. Fix: Upload the key that was used to make this certificate's CSR, decrypted. Where the CSR was generated on the appliance, do not upload a key at all — the appliance still holds the private key and only needs the signed certificate. Verify: expected · The upload is accepted.
check 4 fails	`no-san` (warning)	Browsers stopped honouring the common name years ago. A certificate with no subjectAltName is refused here rather than installed and discovered broken by every user at once. Fix: Re-issue the certificate with the appliance's hostnames and addresses as SANs. Verify: expected · The upload is accepted.
check 5 fails	`chain-out-of-order` (info)	TLS requires the chain in issuing order. A chain assembled by concatenating files in alphabetical order is the usual way this happens, and it fails on some clients and works on others, which makes it hard to see from the outside. Fix: Order the chain leaf-side first — the issuer of the certificate, then its issuer, and so on. Verify: expected · The upload is accepted.
check 6 fails	`reconcile-in-flight` (info)	Installing a certificate is not only a database write: the host has to be reconfigured and the front end restarted. Two overlapping reconciles could leave the appliance serving neither certificate, so the second is refused. Fix: Wait for the pending reconcile to finish, then upload again. Verify: expected · No reconcile is pending and the new certificate is served. If that does not apply: Attach the support bundle and the output of `openssl x509 -noout -text` for the certificate you are uploading. Never attach the private key.

Directory (AD / LDAP)

The directory cannot be reached or bound

Local accounts keep working throughout, and content policy never consults the directory — what stops is directory sign-in and membership updates.

Runbookldap

The directory cannot be reached or bound

You might see: LDAP url, bind DN and base DN must be configured · LDAP bind password file unreadable · Directory authentication is unavailable · The directory server could not be reached, so directory sign-in is temporarily unavailable.

Before you start

Has a directory connection been saved on this appliance at all?expected · All three are present.If not, go to /administer/settings/ldap first.

Checks

1
Read the three fields the connection cannot be attempted without
expected · All three are non-empty.
2
Check where the bind password is coming from
expected · One of the three resolves to a password the directory accepts.
3
If you are testing a URL you just typed, check whether you also re-entered the password
expected · Either the URL is unchanged, or you typed the bind password into the test form.
4
Check that the directory host answers from the appliance
expected · The port answers and, for LDAPS, the certificate chain validates.

Causes and fixes

Reached when	Cause	Explanation, fix and verification
check 1 fails	`incomplete-config` (warning)	The gateway will not guess a base DN or bind as anonymous. A saved configuration missing any of URL, bind DN or base DN is refused at validation, which is why the failure appears instantly and identically for a wrong password and an empty field. Fix: Fill all three on Settings → AD / LDAP and save, then run the connection test. See AD / LDAP. Verify: expected · It succeeds.
check 2 fails	`bind-password-unreadable` (critical)	Where the password comes from a mounted secret file, the gateway reads it at bind time. A file the container cannot read produces an explicit "unreadable" refusal rather than a wrong-password error — the two need different fixes. Fix: Correct the mount and the file's permissions so the gateway container can read it, or store the password through the dashboard instead. The stored password is encrypted with a key derived from `JWT_SECRET`, so changing that secret invalidates it and it must be re-entered. Verify: expected · It succeeds.
check 3 fails	`password-not-sent-to-new-url` (info)	The saved bind password is write-only and is only sent to the host it was saved against. Testing a different URL without retyping it would hand the stored secret to a host you just named, so the gateway refuses instead. Fix: Type the bind password into the test form alongside the new URL. Verify: expected · It succeeds.
check 4 fails	`directory-not-reachable` (critical)	While the directory is unreachable, directory sign-in fails and group membership stays frozen at its last synced state. Local accounts keep working, and content policy and enforcement are unaffected — they never consult the directory. Fix: Open the path from the appliance to the domain controller, or point the URL at a controller that is reachable from this subnet. For LDAPS, install the issuing CA on the appliance so the handshake completes. Verify: expected · Both succeed. If that does not apply: Attach the support bundle and the exact test message. Include whether the URL is ldap:// or ldaps:// and which controller it names.

A sync scope will not run, or imports the wrong people

On a partial directory answer the sync imports and updates what it saw and removes nobody, because removing on incomplete data deprovisions people whose entries were simply not returned.

Runbookldap

A directory sync scope will not run, or imports the wrong people

You might see: Sync scope not found · This sync scope is disabled. Re-enable it before syncing. · Sync scope is mapped to a CID tenant group that no longer exists · Sync scope would assign a role which cannot be granted from a directory scope

Before you start

Does the directory connection test succeed?expected · The test succeeds.If not, follow the ldap-bind-fails runbook instead.

Checks

1
Read the scope's state before anything else
expected · The scope is enabled.
2
Check the CID tenant group the scope maps into
expected · The target group exists.
3
Read the role the scope assigns
expected · The role is one of those the message lists as allowed.
4
Read the result counts of the last run
expected · The run completed rather than reporting a partial answer.

Causes and fixes

Reached when	Cause	Explanation, fix and verification
check 1 fails	`scope-disabled` (info)	A disabled scope is skipped rather than run and failed, so membership simply stops changing. Nothing in the product degrades; it just stops tracking the directory. Fix: Re-enable the scope, then run it once by hand and read the counts. Verify: expected · It completes and reports counts.
check 2 fails	`target-group-deleted` (warning)	The scope names a CID tenant group by id. Deleting that group leaves the scope pointing at nothing, and the sync refuses rather than creating a replacement group and quietly changing who can see what. Deleting a group that is a sync target is refused for the same reason. Fix: Re-point the scope at an existing group, or clear the mapping, then sync again. See Tenant groups. Verify: expected · It completes without naming a missing group.
check 3 fails	`role-not-grantable` (warning)	A directory group is an authorisation input, not an authorisation decision. Roles that administer the appliance itself cannot be handed out by editing a group in AD, so the scope is refused at save time rather than silently downgrading the role. Fix: Set the scope's role to one of those the refusal lists, and grant the others in CID directly. Verify: expected · It saves.
check 4 fails	`partial-directory-answer` (warning)	When the directory answers only part of a query, the sync imports and updates what it saw and removes nobody. Removing on a partial answer would deprovision people whose entries simply were not returned, which is worse than leaving membership stale. Fix: Fix the directory-side limit — usually a server-side size or time limit on the search, or a base DN broader than the controller will page through — then run the scope again. Verify: expected · The run completes without the partial-answer warning. If that does not apply: Attach the support bundle and the scope's base DN, filter and reported counts.

Providers

No providers are in the catalogue

Chat cannot resolve a model, so every completion fails. The catalogue is data, not code.

Runbookproviders

No LLM providers are in the catalogue, so every completion fails

You might see: No LLM providers are in the catalogue · chat cannot resolve a model · the model dropdown is empty · every completion fails with a model error

Before you start

Has the setup wizard completed? Before it has, the wizard's own provider step is the place to look.diagnostics · appliance.setupComplete = trueIf not, follow the setup-providers-step-empty runbook instead.

Checks

1
Verify the gateway can reach the database, since the catalogue is read from it rather than hard-coded
diagnostics · datastores.postgres.reachable = true
2
Verify no migrations are pending, so the provider and model tables have their current shape
diagnostics · datastores.postgres.migrationsPending
3
Ask the gateway for the model catalogue
GET /models

Causes and fixes

Reached when	Cause	Explanation, fix and verification
check 3 fails	`catalogue-never-seeded` (critical)	Provider and model rows come from the database seeder, not from code. A box seeded with seeding switched off entirely has an empty catalogue, and chat cannot resolve a model for any request. Nothing is corrupt: the rows were never inserted. Fix: Run the seeder once against this database with sample accounts disabled, so the catalogue lands without creating demo users, then reload. The seed inserts providers and models only; credentials stay yours to add. Verify: GET /models
check 1 fails	`database-unreachable` (critical)	The gateway cannot reach the database, so every list is empty rather than erroring visibly. The model catalogue is simply the list whose emptiness stops the product working. Fix: Follow PostgreSQL or Redis is unreachable, then re-check the catalogue. Verify: diagnostics · datastores.postgres.reachable = true
check 2 fails	`schema-behind` (critical)	Pending migrations mean the provider or model tables are missing columns the current code selects, so the query fails or returns nothing and the catalogue reads as empty. Fix: Apply the pending migrations and restart the gateway. See Migrations are pending. Verify: diagnostics · datastores.postgres.migrationsPending

A provider credential will not save, or its test fails

Three unrelated refusals share this screen: the endpoint validator, the assignment rule, and the provider refusing the key itself. Only the third is about the key.

Runbookproviders

A provider credential will not save, or its test fails

You might see: Invalid Anthropic API key · Test failed to run · Failed to create credential · Failed to update credential

Before you start

Does the provider you are configuring appear in the provider list at all?expected · The provider you want is listed.If not, follow the providers-catalog-empty runbook instead.

Checks

1
For a credential that carries an endpoint (Azure OpenAI, or any self-hosted endpoint), read the exact rejection text
expected · The endpoint is an absolute http(s) URL that does not resolve into link-local space.
2
Check what the credential is assigned to
expected · Exactly one of them is set.
3
Check whether the appliance can reach the provider at all before blaming the key
diagnostics · appliance.egress
4
Read what the credential test actually reported
expected · The test succeeds.

Causes and fixes

Reached when	Cause	Explanation, fix and verification
check 1 fails	`endpoint-refused-by-validator` (warning)	The gateway validates every outbound endpoint before it will store it (`config/outbound-url.ts`): it must parse as a URL, it must be http or https, and it must not resolve into link-local or cloud instance-metadata space. The last rule is an SSRF guard, and it is the one that surprises people pointing CID at a model server on the same host. Fix: Give the endpoint as an absolute `https://host[:port]/path` URL that resolves to a routable address. For a model server beside the appliance, use its container or LAN name rather than a link-local address. Verify: expected · The form accepts it.
check 2 fails	`assignment-not-exclusive` (info)	A credential belongs to one tenant or to one tenant group, never to both — the same XOR constraint the database enforces on `ai_credentials`. A form that carries both values is refused before anything is written. Fix: Clear whichever of the two assignments you did not mean, then save. Verify: expected · The form accepts it.
check 3 fails	`cannot-reach-provider` (critical)	The provider call leaves the appliance like any other outbound request. Where egress is blocked or must traverse a proxy that is not configured, the provider never answers and the failure is reported as a credential problem. Fix: Follow The appliance cannot get out. Verify: diagnostics · appliance.update.channelReachable = true
check 4 fails	`key-rejected-upstream` (critical)	The provider itself refused the key. The usual reasons are a key from a different organisation or project, a key that has been rotated at the provider, or an Azure deployment whose endpoint and key belong to different resources. Fix: Re-issue the key at the provider and paste it again. Keys are stored as AES-256-GCM ciphertext and are never readable back through the dashboard, so retyping is the only way to confirm what is stored. For Azure OpenAI, set the endpoint URL on the same credential — the provider cannot be called without it. Verify: expected · The test succeeds. If that does not apply: Attach the support bundle and the exact test message; do not attach the key.

A chat request fails before the model is reached

Model row, then credential, then provider call — a failure at each step reads the same to the user and needs a different fix.

Runbookchat

A chat request fails before the model is reached

You might see: Provider 'openai' not found · Model 'gpt-4o' not found · Model 'gpt-4o' is not active · Model not found for provider

Before you start

Is the request failing outright, rather than being rejected or masked by the content policy?expected · The message names a provider, model or credential.If not, follow the chat-content-blocked runbook instead.

Checks

1
Check that the provider and model catalogue is populated at all
expected · Providers and models are listed.
2
Read whether the model the caller named exists and is active
expected · The model is listed and active.
3
Check that a credential resolves for this caller and provider
expected · One active credential resolves.
4
Test the credential that resolves
expected · The test succeeds.

Causes and fixes

Reached when	Cause	Explanation, fix and verification
check 1 fails	`catalogue-empty` (critical)	Providers and models are seeded data, not code. With no rows, no model id can resolve and every completion fails at the first lookup. Fix: Follow No providers are in the catalogue. Verify: expected · Providers and models are listed.
check 2 fails	`model-inactive-or-absent` (warning)	The gateway resolves the model row before it resolves a credential, and it refuses an inactive model rather than silently substituting one. A model deactivated in the catalogue therefore breaks every client still naming it, including clients that worked yesterday. Fix: Re-activate the model under Models, or change the client to a model that is active. A client that hard-codes a model id needs that id to stay active. Verify: expected · The completion starts streaming.
check 3 fails	`no-credential-resolves` (critical)	Credentials resolve tenant-first and fall back to the tenant's group. A tenant in no group, or a group whose credential was deactivated, resolves to nothing and the request fails before any provider call is made. Fix: Add an active credential for that provider on the tenant, or add the tenant to a group that already holds one. See Credentials. Verify: expected · The completion starts streaming.
check 4 fails	`credential-rejected` (critical)	A credential resolves and the provider refuses it, so the failure surfaces on the chat path rather than on the credential page where it was configured. Fix: Follow A provider credential will not save, or its test fails. Verify: expected · Both succeed. If that does not apply: Attach the support bundle and the request's model id, tenant and timestamp.

A request is refused for being too frequent, or a quota is exhausted

A gateway throttle, an API-key quota and the provider's own refusal look alike and have completely different remedies.

Runbookproviders

A request is refused for being too frequent, or a quota is exhausted

You might see: Too many requests, please try again later. · Too many help requests, please try again in a few minutes. · Too many unlock requests, please try again later. · Too many attestation requests

Checks

1
Establish which limit refused the call — the gateway's own throttle, an API-key quota, or the provider's
expected · You can say which of the three it is.
2
For a quota message, read the key's configured request and token quotas
expected · The key has headroom left in the current window.
3
Check that Redis is reachable, because the throttles keep their counters there
diagnostics · datastores.redis.reachable = true

Causes and fixes

Reached when	Cause	Explanation, fix and verification
check 1 fails	`cannot-attribute` (info)	A gateway throttle and a provider refusal look alike to the caller and have completely different fixes: one is a setting on this appliance, the other is an account limit at the provider that no CID setting can raise. Fix: Find the request in Event Logs and read its route and status. A gateway throttle refuses before any outbound call, so no provider call is logged for it. Verify: expected · The route tells you which limiter fired.
check 2 fails	`key-quota-exhausted` (warning)	Gateway API keys carry optional request and token quotas, and the key is refused once either is used up. This is a deliberate cap on the key, not a fault. Fix: Raise the quota on the key, issue a second key for the workload, or wait for the window to roll. See Access keys. Verify: expected · The request is accepted.
check 3 fails	`redis-unreachable` (warning)	Redis holds the throttle counters and the scheduled work. When it is unreachable, rate limiting degrades open rather than closed, so a burst of refusals while Redis is down comes from somewhere else. Fix: Follow PostgreSQL or Redis is unreachable. Verify: diagnostics · datastores.redis.reachable = true

A gateway API key is refused

Keys are hashed and cannot be recovered, only replaced. A group key borrows the identity of the group's earliest-added member, so emptying the group stops it resolving.

Runbookaccess

A gateway API key is refused

You might see: Invalid API key · Invalid or expired API key · API key is not active · API key has expired

Before you start

Is the caller presenting a gateway API key (a `cid_key_` bearer token) rather than a dashboard sign-in?expected · It is `Bearer cid_key_…`.If not, follow the sign-in-refused runbook instead.

Checks

1
Find the key in the estate
expected · The key exists.
2
Read the key's status and expiry
expected · The key is active and either has no expiry or expires in the future.
3
Check what identity the key resolves to
expected · The key is assigned to a tenant, or to a group that has at least one member.

Causes and fixes

Reached when	Cause	Explanation, fix and verification
check 1 fails	`key-not-present` (warning)	Keys are stored as an unsalted SHA-256 hash of 32 random bytes, so a key that was deleted or never created cannot be recognised and cannot be recovered from the database. Fix: Issue a new key under Access Keys and reconfigure the client with it. Verify: expected · The call is accepted.
check 2 fails	`key-inactive-or-expired` (warning)	A key carries an optional `expires_at` (NULL means never) and an active flag. Both refuse the call at the guard, before any route logic runs, which is why the refusal looks identical to a typo in the key. Fix: Re-activate the key, extend its expiry, or issue a replacement and rotate the client. Verify: expected · The call is accepted.
check 3 fails	`group-key-has-no-member` (warning)	A group key does not have an identity of its own — it borrows one from the group's earliest-added member. Empty the group, or remove that member, and the key resolves to no tenant and is refused. Removing that user also transfers the key's identity rather than revoking the key. Fix: Add a member to the group, or reassign the key to a tenant directly. Treat a group key as carrying a person's identity until service identities ship. Verify: expected · The call is accepted and the resulting records are attributed to the expected tenant. If that does not apply: Include the key prefix (never the key), the assignment, and the exact 401 body. Do not paste the key into a ticket — it cannot be recovered from the hash, so a leaked key must be revoked.

Chat and detections

A message is rejected, masked or flagged by the content policy

A decision is not an error. Every decision writes a detection naming the engine, the label and the action — start from that record rather than the user's paraphrase.

Runbookchat

A message is rejected, masked or flagged by the content policy

You might see: Document rejected due to policy violation · my prompt came back with names replaced by placeholders · the assistant refused an ordinary business question · a customer record was masked and the model could not answer

Before you start

Did the request produce a policy decision, rather than an error naming a provider, model or credential?expected · The request appears in All Detections with a decision.If not, follow the chat-completion-fails runbook instead.

Checks

1
Find the request in the detection record
expected · One detection explains the outcome.
2
Read which engine produced the match
expected · You can name the engine.
3
Read the filter that fired and the action it carries
expected · The action matches what the user experienced.
4
Decide whether the match was correct
expected · The matched span really is what the rule is for.

Causes and fixes

Reached when	Cause	Explanation, fix and verification
check 1 fails	`no-detection-recorded` (warning)	Every policy decision writes a detection. A refusal with no detection behind it did not come from the content policy — it came from the provider, from a role or licence refusal, or from the model's own refusal to answer. Fix: Read the response body and the event log entry for that request. A model refusing on its own terms is not a CID decision and no CID setting changes it. Verify: expected · Either a detection appears, or the refusal is attributable elsewhere.
check 2 fails	`cannot-attribute-engine` (info)	PII, toxicity and injection detection run in parallel and each has its own thresholds. Tuning the wrong one changes nothing and weakens a control you did not mean to touch. Fix: Use the detection's engine and label to decide where to look: PII detection for entities, the content-safety pipeline for toxicity and injection. Verify: expected · You can name the engine and its label.
check 3 fails	`unexpected-action-priority` (info)	When several rules match one message, the strongest action wins: REJECT over MASK over FLAG. A rule set to FLAG therefore still produces a rejection when something else on the same message rejects. Fix: Read every detection on that request, not only the first. Change the rule that carries the strongest action. See Filters. Verify: expected · The action matches the rule you changed.
check 4 fails	`false-positive` (warning)	A regex written for one format matches other text that shares its shape, and a model score just above its block threshold produces a confident-looking block on ordinary text. Both are tuning problems, not faults. Fix: For a pattern, narrow it and re-test it on the Filter Testing page before saving — a pattern that fails to compile protects nothing and the filter list marks it broken. For a model label, the block thresholds are properties of the shipped model image rather than runtime settings, so the remedy is a scoped filter override rather than a threshold edit. Verify: expected · The text passes, and the cases the rule exists for still fail. If that does not apply: Attach the detection id and the exact input text if policy allows it to leave the estate. Where it may not, attach the detection id alone — the record holds the matched span.

A user's AI access is locked

Repeated detections fill a sliding window that opens one per-user review. A lock the analyst could not tie back to detections is downgraded to a warning, so a lock that survived has evidence behind it.

Runbookaccess

A user's AI access is locked and they cannot chat

You might see: user_locked · the chat stream ends immediately with a lock notice · Your AI access is not locked · An unlock request is already pending for this lock

Before you start

Does the refusal name a lock, rather than a role, a licence or the content policy?expected · The stream carried `user_locked`.If not, follow the role-or-licence-refusal runbook instead.

Checks

1
Read whether the user actually holds an active lock
expected · One active lock exists for that user.
2
Read the review that produced the lock
expected · The review explains the lock in terms of specific detections.
3
For a user who cannot file an unlock request, read the state of the existing one
expected · Either no request is pending, or the pending one is waiting for a reviewer.
4
After an unlock, confirm the lock is gone rather than only appearing gone
expected · The message goes through.

Causes and fixes

Reached when	Cause	Explanation, fix and verification
check 1 fails	`no-active-lock` (info)	Unlock requests attach to a lock. With no active lock there is nothing to request, and the gateway says so rather than creating an orphan request — a request without its lock would be a user everyone believes is locked and who is not. Fix: Have the user retry the chat. If it still fails, the refusal is not a lock: read the response body for a role, licence or policy decision instead. Verify: expected · The message goes through.
check 2 fails	`verdict-without-evidence` (warning)	The analyst must tie every claim in its reasoning back to a detection or a tool result. Where it cannot, a lock verdict is downgraded to a warning rather than enforced — so a lock you cannot explain from the review is worth reopening rather than defending. Fix: Open the review, read its failed checks, and decide the unlock request on the evidence. A review whose own checks failed is marked unverified on the analyses list. Verify: expected · The decision is recorded against the review.
check 3 fails	`request-already-filed` (info)	One pending request per lock, enforced both by a partial unique index and by the API, and three requests per hour per user. A decided request is final; the user files a new one only after a new lock. Fix: Decide the pending request. A denial leaves the lock in place and the user may send another request; approving it clears the lock. Verify: expected · No request is left pending.
check 4 fails	`lock-cache-not-expired` (info)	The active lock is read through a short-lived Redis cache so that every chat turn does not query Postgres. A lock cleared in the dashboard can therefore still refuse for the length of that cache window. Fix: Wait out the cache window and try one more message before investigating further. Verify: expected · The message goes through. If that does not apply: Attach the lock id, the review id and the support bundle.

A file is refused, or its analysis fails

Format, size and a document parser behind its circuit breaker all refuse in the same place. A chunked upload is refused rather than passed through uninspected.

Runbookperformance

A file is refused, or its analysis fails

You might see: No file uploaded (expected multipart field "file") · Image size exceeds maximum allowed size of 10MB · Unsupported file type. Upload a PDF, DOCX or TXT. · legacy and macro-enabled spreadsheet formats cannot be safely redacted

Before you start

Which surface refused the file — the chat, the document analysis page, the help assistant, or an inspected browser upload?expected · You can name the surface. Each has its own limits.If not, go to /troubleshoot/index first.

Checks

1
Read whether the format is one the product accepts at all
expected · The format is accepted on that surface.
2
Read the size limit for that surface
expected · The file is under the limit named in the message.
3
Check whether the client was doing a chunked or resumable upload
diagnostics · posture.resumableUploadPolicy = "block"
4
Read whether document analysis is switched on for this deployment
expected · The service is enabled.
5
Read the document parser's own health
diagnostics · services[document-parser].healthy = true

Causes and fixes

Reached when	Cause	Explanation, fix and verification
check 1 fails	`format-not-accepted` (info)	Some formats are refused on purpose rather than partially handled. A macro-enabled spreadsheet cannot be redacted safely, and a redaction that silently missed content would be worse than a refusal. Fix: Save the file in an accepted format — a spreadsheet as `.xlsx` without macros, a document as PDF, DOCX or TXT — and upload it again. Verify: expected · It is accepted.
check 2 fails	`over-size-limit` (info)	Each surface caps what it will buffer, because analysis holds the whole file in memory and an unbounded upload is an out-of-memory kill for whichever detection service is largest. Fix: Split or compress the file, or use the document path rather than the image path for large scans. Verify: expected · It is accepted.
no check points here	`resumable-refused-by-policy` (warning)	No path buffers across requests, so a chunked or resumable upload cannot be inspected. Rather than passing it through uninspected, all three paths refuse it by default and the extension records a visible "not inspected" event. Reassembly — buffer by upload id, inspect at finalize, re-fragment — is not implemented. Fix: Have the user upload the file in a single request, or accept the risk explicitly by changing the policy. Do not read a refusal here as a product fault: it is the fail-closed choice working. Verify: diagnostics · posture.resumableUploadPolicy = "block"
check 4 fails	`parser-disabled` (warning)	Document analysis is a separate service and can be switched off, for instance on a minimal development stack. The endpoint answers 503 with a distinct message so it is not confused with a service that is enabled and down. Fix: Enable the document parser and restart the gateway so it re-reads the setting. See File redaction. Verify: expected · The analysis runs.
check 5 fails	`parser-unhealthy` (critical)	The gateway holds a circuit breaker in front of the parser: once it has failed repeatedly the gateway stops calling it and answers "temporarily unavailable" immediately, which is why the failure can look instant. Fix: Follow One service is unreachable or reports itself unhealthy. Verify: diagnostics · services[document-parser].healthy = true If that does not apply: Attach the support bundle and the file's type and size. Do not attach the document itself unless policy allows it to leave the estate.

Updates

The System Updates page shows 0.0.0, or the buttons return 500

A missing version stamp, an unset channel, an unreachable channel and a stalled updater all surface on the same page and look alike.

Runbookupdates

The System Updates page shows 0.0.0, or the update buttons return 500

You might see: System Updates shows version 0.0.0 · current version 0.0.0 · update check returns 500 · install update → 500

Before you start

Has the setup wizard completed? The update page is not meaningful before it has.diagnostics · appliance.setupComplete = trueIf not, go to /install/setup-wizard first.

Checks

1
Read the product version the appliance reports
diagnostics · appliance.version = "0.0.0"
2
Verify an update channel is configured
diagnostics · appliance.channel
3
Verify the release channel answered on the last attempt
diagnostics · appliance.update.channelReachable = true
4
Check whether an update or host-repair intent is stuck waiting for the host updater
diagnostics · appliance.update.pendingIntent
5
Read the host-repair status, which is what the page renders alongside the version
GET /admin/system-update/host-repair

Causes and fixes

Reached when	Cause	Explanation, fix and verification
check 1 fails	`version-file-missing` (warning)	The appliance reads its installed version from a file the image build writes. When that file is absent or unreadable the version falls back to 0.0.0, and every comparison against the channel manifest then looks like a downgrade — which is why the page can show 0.0.0 and simultaneously refuse to offer an update. Fix: Run the host repair from Settings → System Updates, which rewrites the appliance version file, then reload the page. If host repair is unavailable, the version can be restored by re-running the installer's finalisation step; see Update the appliance. Verify: diagnostics · appliance.version = "0.0.0" If that does not apply: Attach the support bundle. A 0.0.0 that survives host repair means the image was built without its version stamp.
check 2 fails	`manifest-url-unset` (warning)	No release channel is configured, so the update check has no manifest to fetch. The API rejects the call with a 400 naming the missing manifest URL, which the page surfaces as a failed button rather than as a configuration prompt. Fix: Set the update channel under Settings → System Updates. On an air-gapped appliance, choose the offline channel and upload the update bundle instead of pointing at a URL. Verify: diagnostics · appliance.channel
check 3 fails	`channel-unreachable` (warning)	The channel is configured but the manifest fetch failed — no DNS, no egress, or an outbound proxy that is required and not configured. The fetch failure surfaces as a 500 from the update endpoint. Fix: Fix outbound access to the channel host, or switch the appliance to offline updates. Read the DNS and egress probes in the diagnostics snapshot separately: resolution succeeding does not mean the HTTPS fetch succeeds, and a proxy-only network needs the proxy configured explicitly. Verify: diagnostics · appliance.update.channelReachable = true
check 4 fails	`updater-daemon-not-running` (critical)	An intent is queued but nothing is executing it. The gateway only records the intent; the privileged updater daemon on the host performs the work. When that daemon is not running, intents accumulate and every subsequent action appears to hang or fail. Fix: Start the updater daemon on the appliance host and confirm the queued intent moves out of the queued state. Do not queue further intents until it does; a backlog makes the failure harder to read. Verify: diagnostics · appliance.update.pendingIntent If that does not apply: Attach the support bundle plus the updater daemon's own log from the host.
check 5 fails	`host-agent-absent` (warning)	The host-repair endpoint cannot answer because the host agent has never written its inventory. The page then has no host-side data to render beside the version, and the combination reads as a broken page. Fix: Run the host agent's inventory job on the appliance and reload the page. This is expected on a non-appliance deployment, where there is no host agent at all and the update page is not the supported upgrade path. Verify: GET /admin/system-update/host-repair

The appliance cannot say which release it runs

Every comparison against the channel manifest is meaningless until the version stamp is restored, and so is every support answer.

Runbookupdates

The appliance cannot say which release it is running

You might see: This box cannot say which release it is running · System Updates shows version 0.0.0 · current version 0.1.0 on a box that is not 0.1.0 · the update page offers nothing and shows no version

Before you start

Has the setup wizard completed?diagnostics · appliance.setupComplete = trueIf not, go to /install/setup-wizard first.

Checks

1
Read the product version the appliance reports
diagnostics · appliance.version = "0.1.0"
2
Read the host-repair status, which is what rewrites the version stamp
GET /admin/system-update/host-repair

Causes and fixes

Reached when	Cause	Explanation, fix and verification
check 1 fails	`no-version-stamp` (warning)	The appliance reads its installed version from files the image build and the updater write. When none of them can be read the version falls back to a placeholder, and every comparison against the channel manifest is then meaningless — which is why the page can show a version that is obviously wrong and simultaneously refuse to offer an update. Support answers are guesses until this is fixed. Fix: Run the host repair from Settings → System Updates, which rewrites the appliance version file, then reload the page. A placeholder that survives host repair means the image was built without its version stamp. Verify: diagnostics · appliance.version = "0.1.0" If that does not apply: Attach the support bundle and the version named in the installer's boot menu entry.
check 2 fails	`host-agent-absent` (warning)	The host-repair endpoint cannot answer because the host agent has never written its inventory, so the update page has no host-side data to render and nothing can rewrite the version file. On a non-appliance deployment there is no host agent at all and this page is not the supported upgrade path. Fix: Run the host agent's job on the appliance and reload the page. If this is a compose deployment rather than an appliance, update it by pulling the release images instead. Verify: GET /admin/system-update/host-repair

The update channel is unreachable

Expected on a deliberately air-gapped appliance, which updates from a signed offline bundle. On a connected one it is an egress problem wearing an update-shaped mask.

Runbookupdates

The update channel is not reachable from the appliance

You might see: The update channel host is not reachable from inside the gateway · Check for updates does nothing · update check returns 500 · the appliance never finds a new version

Before you start

Is this appliance meant to update online, rather than from an offline bundle?diagnostics · appliance.channel = "offline"If not, go to /install/updates first.
Has the setup wizard completed? The update page is not meaningful before it has.diagnostics · appliance.setupComplete = trueIf not, go to /install/setup-wizard first.

Checks

1
Verify an update channel is configured at all
diagnostics · appliance.channel
2
Read whether the channel answered on the last attempt
diagnostics · appliance.update.channelReachable = true
3
Read the name-resolution results for the hosts the appliance must reach
diagnostics · appliance.dns
4
Read whether an outbound proxy is configured, on a network that requires one
diagnostics · appliance.proxy.configured = true

Causes and fixes

Reached when	Cause	Explanation, fix and verification
check 1 fails	`channel-unset` (warning)	No release channel is configured, so the update check has no manifest to fetch. The API rejects the call, and the page shows a button that fails rather than a prompt to configure anything. Fix: Set the update channel under Settings → System Updates. On an air-gapped appliance, choose the offline channel and upload a signed bundle instead of pointing at a URL. Verify: diagnostics · appliance.channel
check 2 fails	`channel-blocked` (warning)	The channel is configured and the fetch failed. On an appliance that is deliberately air-gapped this is the expected state and needs no action; on a connected one it means the HTTPS path to the channel host is blocked, intercepted, or requires a proxy the appliance does not have. Fix: Read the egress results in the diagnostics snapshot and fix the test that failed rather than the one you assume failed. See The appliance cannot get out. If the appliance is meant to stay offline, switch it to the offline channel so the failure stops being reported as a fault. Verify: diagnostics · appliance.update.channelReachable = true
check 3 fails	`dns-cannot-resolve-channel` (warning)	The appliance cannot resolve the channel host, so the fetch never reaches the network. DNS succeeding elsewhere proves nothing: the appliance uses this network's resolver, not the browser's. Fix: Set a resolver that answers for the channel host and re-run the check. See Network prerequisites. Verify: diagnostics · appliance.dns
check 4 fails	`proxy-missing` (warning)	This site sends everything through a proxy and the appliance has none configured, so the fetch is attempted directly and times out. A timeout looks like a dead channel rather than a missing setting. Fix: Configure the outbound proxy on the appliance and restart the gateway container so it re-reads the setting. Verify: diagnostics · appliance.proxy.configured = true

An update bundle will not upload or install

Resumable uploads fail on offsets and expiry rather than on the release, and an online install is refused outright unless the update host is pinned.

Runbookupdates

An update bundle will not upload or install from the dashboard

You might see: unknown or expired upload · offset mismatch · incomplete upload: have N of M bytes · bundle filename must end with .cidupd

Before you start

Does the System Updates page load and report a real version, rather than 0.0.0 or a 500?diagnostics · appliance.versionIf not, follow the update-version-zero-or-500 runbook instead.

Checks

1
Read the bundle's file name
expected · The file ends in `.cidupd`.
2
For a resumed upload, check that the server still holds the partial file
expected · The status call returns an offset.
3
For an offset mismatch, compare what the client thinks it sent with what the server holds
expected · The client resumes from the offset the server reports.
4
For "incomplete upload", compare the assembled size with the declared size
expected · The two sizes agree.
5
For an online install, read the update host allowlist
expected · The allowlist names the host in the manifest URL, over https.

Causes and fixes

Reached when	Cause	Explanation, fix and verification
check 1 fails	`wrong-extension` (info)	The file name is normalised to a safe character set and must end in `.cidupd`. This is the first gate and it fires before anything is read, which is why a corrupted bundle and a renamed one look the same. Fix: Restore the original file name and upload again. Verify: expected · The upload starts.
check 2 fails	`upload-expired` (info)	A resumable upload is a partial file in the incoming directory plus a small metadata file. Once either is cleaned up there is nothing to resume, and the server reports it as unknown rather than starting a silent second copy. Fix: Start the upload again from the beginning. Verify: expected · It reports a growing offset.
check 3 fails	`offset-drifted` (info)	The server refuses an append whose offset does not match the bytes it holds, so a client that retried a chunk cannot duplicate or skip one. The refusal carries the real offset. Fix: Re-query the upload status and resume from the offset the server reports. Verify: expected · The offset advances to the full size.
check 4 fails	`transfer-truncated` (warning)	A short bundle would fail signature verification later, on the host, in the middle of a maintenance window. Refusing at completion keeps that failure in the browser. Fix: Upload again over a stable path. A proxy that buffers or truncates large bodies is the usual cause; upload from a machine on the same network as the appliance to rule it out. Verify: expected · It reports the full size and hands back a bundle path.
check 5 fails	`online-host-not-allowed` (warning)	An online install fetches a manifest over the network, so an unpinned host would be an SSRF primitive with administrator privileges. An empty allowlist used to mean "any https host" and empty is the shipped default, so the guard was off wherever nobody configured it — it now refuses instead. Fix: Set `SYSTEM_UPDATE_ALLOWED_HOSTS` to the update host and restart the gateway, or install from an uploaded bundle instead. See Updates. Verify: expected · It is queued. If that does not apply: Attach the support bundle and the request id from the failed intent. Do not attach the bundle itself.

The updater refuses the release, or rolls it back

Every refusal before the smoke test happens with nothing changed. The one that surprises people is the rollback check: pruned images mean there is no way back, so the updater will not go forward.

Runbookupdates

The updater refuses the release, or stops part-way through installing it

You might see: signature missing · bundle not found · compose file missing from release · cannot jump from X to Y: this release requires at least

Before you start

Did the bundle upload complete, or was the install queued from a pinned online host?expected · An intent exists with a bundle path or a manifest URL.If not, follow the update-bundle-upload-rejected runbook instead.

Checks

1
Read whether the failure is the release's signature or the appliance's trust anchor
diagnostics · appliance.license.trustAnchorPresent = true
2
Read the version this appliance runs against the release's minimum
diagnostics · appliance.version
3
Check that the bundle carries everything its manifest references
expected · No file is reported missing or mismatched.
4
Check that the images of the release currently installed are still present
expected · Every current-release image inspects successfully.
5
Read the database pre-flight probes
expected · Every probe reports clean.
6
For an update that installed and then rolled back, read the smoke result
expected · The smoke test passed.

Causes and fixes

Reached when	Cause	Explanation, fix and verification
check 1 fails	`no-trust-anchor` (critical)	Every release is verified against the public keys baked into the appliance. With no anchor present nothing verifies, and the same appliance will also refuse a licence for the same reason. Fix: Follow The appliance has no licence trust anchor. Verify: diagnostics · appliance.license.trustAnchorPresent = true
check 2 fails	`version-unknown` (critical)	The updater compares the running version against the release's declared minimum. An appliance that cannot say what it runs cannot be told whether the jump is legal, and every comparison against the manifest is meaningless. Fix: Follow The appliance cannot say which release it runs. Verify: diagnostics · appliance.version
check 3 fails	`release-incomplete` (warning)	A release is a manifest plus the files it names. Extraction refuses members whose paths escape the work directory, and verification refuses a manifest whose files are absent or whose checksums differ — before anything on the appliance is touched. Fix: Obtain the bundle again and re-upload it. A bundle that fails the same way twice is a build problem, not a transfer problem — report it with the release version. Verify: expected · Verification passes.
check 4 fails	`current-images-pruned` (critical)	Rollback works by re-pinning the images of the release you are on. Once those images have been pruned there is no way back, so the updater refuses to go forward rather than taking a one-way trip. Pinned images carry a keep-tag for exactly this reason. Fix: Restore or re-pull the current release's images, then retry. Take a snapshot of the appliance before retrying, so a rollback exists at the hypervisor level even if it does not exist at the image level. Verify: expected · The pre-install check passes.
check 5 fails	`preflight-refusal` (warning)	The probes exist to stop a migration failing half-way through on data it cannot handle. Nothing has changed at the point the refusal fires, so the appliance is still fully serviceable. Fix: Apply the fix the probe names — it is written into the refusal — then queue the install again. Do not force past a probe. Verify: expected · The probes report clean.
check 6 fails	`smoke-failed` (critical)	The post-update smoke test is the last gate. Its failure means the new release came up and did not work, which is exactly the case rollback exists for. The tail of the failing command is carried in the message. Fix: Let the updater roll back, then read the smoke output and the container logs for the service it exercised. A pgdata snapshot is taken before the migration; restore refuses if that snapshot is missing, so check that the state directory is writable. Verify: expected · The appliance is back on the previous release and healthy. If that does not apply: Attach the support bundle, the updater journal for the request id, and the release version you were installing.

A host-side repair or reconfigure step fails

The failure mode that costs the most is the quiet one: the host write fails, the handler logs and continues, and the dashboard reports success.

Runbookupdates

A host-side repair or reconfigure step fails

You might see: host command failed · host write failed · cannot determine own image · a certificate was applied in the dashboard and the host never changed

Before you start

Does this deployment run the host agent at all?diagnostics · hostIf not, go to /troubleshoot/diagnostic-snapshot first.

Checks

1
Check whether the agent can identify its own image
expected · The agent resolves its own image, or `CID_HOST_EXEC_IMAGE` pins one.
2
For "host write failed", read which path it was writing
expected · The path is writable and the filesystem has room.
3
Read free disk on the host
diagnostics · host
4
Check the shared state directory, which several features fail through together
expected · The directory exists and the gateway can write it.

Causes and fixes

Reached when	Cause	Explanation, fix and verification
check 1 fails	`cannot-resolve-own-image` (critical)	Host operations run in a privileged helper container built from the agent's own image, so that what executes on the host is exactly the code the signed release shipped. When the agent cannot inspect its own container it refuses rather than falling back to something unpinned. Fix: Confirm the agent's container can reach the Docker socket, or pin the helper image explicitly with `CID_HOST_EXEC_IMAGE`. Restart the agent afterwards. Verify: expected · It completes.
check 2 fails	`host-path-not-writable` (warning)	The agent writes host files through a chroot into the host root. A path that is read-only, immutable, or on a filesystem with no space produces the same "host write failed" with a different tail — read the tail. Fix: Make the path writable, or free space, then trigger the operation again. Verify: expected · It reports success.
check 3 fails	`disk-full` (critical)	A full root filesystem breaks host writes, the state directory and any update that needs room for a second copy of every image — usually all at once, which makes the first symptom you notice arbitrary. Fix: Follow A filesystem is nearly full. Verify: expected · The filesystem has headroom.
check 4 fails	`state-dir-not-writable` (critical)	Several features write intents into one shared directory and log-and-continue when the write fails. The visible symptom is therefore a change that appears to succeed in the dashboard and never reaches the host. Fix: Follow Settings the appliance saves do not survive a restart. Verify: expected · The status reports the change applied. If that does not apply: Attach the support bundle. It carries the host agent's own status documents, which is where the failing command and its output are recorded.

Database and schema

Migrations are pending

The endpoints reading the new columns answer 500 while the rest of the product looks healthy.

Runbookdata

Migrations are pending, and the pages that read the new columns fail

You might see: compiled migration(s) have never been applied to this database · one page returns 500 while the rest of the product works · column does not exist · relation does not exist

Before you start

Can the gateway reach the database at all?diagnostics · datastores.postgres.reachable = trueIf not, follow the datastore-unreachable runbook instead.

Checks

1
Read the list of compiled migrations that are not recorded in the database
diagnostics · datastores.postgres.migrationsPending
2
Verify auto-synchronise is off, since it hides an unapplied chain behind a schema that looks right
diagnostics · posture.dbSynchronize = false
3
Verify the gateway is not restarting, which is what a failing boot migration usually produces
diagnostics · containers[nestjs-core].restarts = 3

Causes and fixes

Reached when	Cause	Explanation, fix and verification
check 1 fails	`boot-migration-failed` (critical)	The container entrypoint applies the compiled migration chain on every boot when auto-synchronise is off. A non-empty pending list on a booted appliance therefore means that run failed rather than that it never happened. The product stays up: only the endpoints that select the new columns answer 500, which is why one page breaks and the rest looks healthy. Fix: Read the gateway log from the boot that failed and fix the reason the migration could not apply — a lock held by another connection, an object the migration expected to be absent, or a permission on the database role. Then restart the gateway so the chain runs again. Do not hand-edit the migration history table to mark them applied; the columns will still be missing. Verify: diagnostics · datastores.postgres.migrationsPending If that does not apply: Attach the support bundle and the gateway log covering the failed boot.
check 2 fails	`synchronize-masking-the-chain` (critical)	Auto-synchronise shapes the schema from the entity declarations, so the tables look correct while the migration history stays empty. Every boot then drops whatever the entities do not declare, and the pending list never clears. Fix: Turn auto-synchronise off and let the chain apply. See The database schema is being auto-synchronised. Verify: diagnostics · posture.dbSynchronize = false
check 3 fails	`migration-crashloop` (critical)	A migration that throws during boot takes the gateway with it, so the container restarts, tries the same migration, and fails again. The restart count climbs while nothing else changes. Fix: Stop the container, read the log from one failed start end to end, and resolve the specific error before starting again. Restarting into the same failure produces no new information. Verify: diagnostics · containers[nestjs-core].restarts = 3 If that does not apply: Attach the support bundle and the name of the migration that fails.

PostgreSQL or Redis is unreachable

A database that cannot be reached empties every list rather than erroring; a Redis that cannot be reached degrades rate limiting open and stops scheduled work.

Runbookdata

PostgreSQL or Redis is unreachable from the gateway

You might see: Postgres is unreachable · Redis is unreachable · ECONNREFUSED on the database port · every list in the dashboard is empty

Before you start

Does the gateway itself answer, so the failure is between it and the datastore rather than in front of it?diagnostics · services[nestjs-core].reachable = trueIf not, follow the first-boot-no-dashboard runbook instead.

Checks

1
Read whether the gateway can reach PostgreSQL
diagnostics · datastores.postgres.reachable = true
2
Read the state of the database container
diagnostics · containers[postgres].state = "running"
3
Read whether the gateway can reach Redis
diagnostics · datastores.redis.reachable = true
4
Check that the filesystem holding the database has space, since PostgreSQL stops accepting writes when it does not
diagnostics · host.disks[/].freeGb = 2

Causes and fixes

Reached when	Cause	Explanation, fix and verification
check 1 fails	`postgres-down` (critical)	Everything the product persists is in PostgreSQL: tenants, credentials, filters, detections, sessions and the event log. When it is unreachable the gateway answers, and every database-backed page is empty or errors, so the failure reads as many unrelated faults at once. Fix: Start the database container and wait for its healthcheck to pass, then reload. If the container is running and still unreachable, confirm the host, port, database name and credentials the gateway uses match what the database is actually serving — a changed password in the environment file is the common cause after a redeploy. Verify: diagnostics · datastores.postgres.reachable = true
check 2 fails	`postgres-container-not-running` (critical)	The container is not in the running state, so nothing is listening. An exited database container after a reboot is usually an out-of-memory kill or a corrupt data directory, and the container log names which. Fix: Read the database container's log, resolve what it reports, and start it again. Do not delete the data volume to make the container start; that discards every detection and audit record on the appliance. Verify: diagnostics · containers[postgres].state = "running" If that does not apply: Attach the support bundle and the database container log.
check 3 fails	`redis-down` (warning)	Redis holds rate-limiting counters and the cluster lock. With it down, rate limiting degrades open and the lock fails closed, so scheduled work stops while request handling continues. The product therefore looks healthy from a browser and is not doing everything it should. Fix: Start the Redis container and confirm the gateway reconnects. Check the memory limit too: a Redis evicted for memory pressure comes back and then disappears again under the same load. Verify: diagnostics · datastores.redis.reachable = true
check 4 fails	`disk-exhausted` (critical)	PostgreSQL refuses writes when its filesystem fills, and the gateway reports that as a datastore failure rather than as a disk problem. Retention settings and an unrotated event log are the usual reasons the disk fills on an appliance that ran fine for months. Fix: Free space and restart the database, then reduce what is being retained. See A filesystem is nearly full. Verify: diagnostics · host.disks[/].freeGb = 2

The schema is auto-synchronised on every boot

Auto-synchronise drops every object the entities do not declare, including the indexes the migrations created. Nothing errors; the appliance just gets slower after each restart.

Runbookdata

The database schema is being auto-synchronised on every boot

You might see: DB_SYNCHRONIZE=true · indexes disappear after a restart · search is slow again after every reboot · the schema changes without a migration

Before you start

Is this a production or customer appliance, rather than a developer box currently iterating on entities?diagnostics · appliance.deploymentMode = "development"If not, go to /develop/getting-started/introduction first.

Checks

1
Read the effective auto-synchronise setting
diagnostics · posture.dbSynchronize = false
2
Verify the migration chain is fully applied, since auto-sync hides an unapplied chain
diagnostics · datastores.postgres.migrationsPending

Causes and fixes

Reached when	Cause	Explanation, fix and verification
check 1 fails	`synchronize-enabled` (critical)	Auto-synchronise makes the schema match the entity declarations on every boot, which means it drops every object the entities do not declare — including the indexes the migrations created. The product keeps working and gets slower after each restart, and the loss is silent because nothing errors. Migrations own the schema on every deployment; only the developer overlay opts into synchronise. Fix: Set `DB_SYNCHRONIZE` to `false` in the appliance's environment file and restart the gateway container. The entrypoint then applies the compiled migration chain on boot, which restores the objects auto-sync removed. Confirm afterwards that no migrations are pending. Verify: diagnostics · posture.dbSynchronize = false
check 2 fails	`chain-never-applied` (critical)	With auto-synchronise on, the migration chain may never have run, so the recorded migration history does not match the schema in front of it. Turning synchronise off without applying the chain leaves a database that neither mechanism owns. Fix: Apply the pending migrations, then restart. On a database that predates the migration chain, baseline it first so the initial migration is recorded rather than re-run. See Migrations are pending. Verify: diagnostics · datastores.postgres.migrationsPending If that does not apply: Attach the support bundle and the output of the migration status command.

Services and capacity

One service is unreachable or reports itself unhealthy

Reachable and unhealthy is the interesting case, and usually means a model that did not load.

Runbookperformance

One service is unreachable or reports itself unhealthy

You might see: hap-guard-v2 is unreachable: no response · ocr-service answered but is not healthy · ml-detector is unreachable · one detector is down and the others are fine

Before you start

Which service is it? The finding and the health matrix both name one — the answer differs for a detector, the OCR service and the analyst.expected · You can name one service, for example hap-guard-v2 or ocr-service.If not, go to /troubleshoot/health-matrix first.
Is this a single service, rather than every service at once?diagnostics · services[nestjs-core].reachable = trueIf not, follow the first-boot-no-dashboard runbook instead.

Checks

1
Read whether the named service answered its health endpoint at all
expected · reachable is true. When it is false the failure is network or container level, not model level.
2
Read the container state for the named service
expected · state is running, restarts are low and oomKilled is false.
3
For a service that answers and reports itself unhealthy, read the error it returns
expected · The error names what did not initialise — a model file, a device, a dependency.
4
For the analyst and the MCP server, check what they depend on before blaming them
expected · The dependency is healthy, so the unhealthy verdict is about this service.

Causes and fixes

Reached when	Cause	Explanation, fix and verification
check 1 fails	`not-reachable` (critical)	The health endpoint did not answer at all, so the problem is below the application: the container is not running, it is still starting, or the gateway is pointed at the wrong address. A service that is not deployed at all is absent from the snapshot rather than unreachable, so an unreachable row means something that should be there is not answering. Fix: Start the container and wait for its healthcheck. If it is running, check that the URL the gateway uses matches the port the container listens on — several services listen on a port that differs from the host publication, and a hand-edited environment file is the usual way that drifts. Verify: expected · reachable and healthy are both true.
check 2 fails	`container-not-running` (critical)	The container is exited, restarting or dead. Detection services load their models into memory at start, so a machine under memory pressure kills these first and the failure looks like a broken service rather than an undersized appliance. Fix: Follow A container was killed for running out of memory when oomKilled is true, and A container keeps restarting otherwise. Verify: expected · state is running and the restart count is stable.
check 3 fails	`model-did-not-load` (warning)	The service answers and reports itself unhealthy, which almost always means a model that did not load. On an air-gapped appliance the usual reason is an image built without its model files: the service starts, discovers the files are absent, and reports the failure rather than crashing. Fix: Read the error the health endpoint returns, then follow the runbook for that service. For the OCR service, see The OCR service has no models on an air-gapped appliance. Restarting a service whose model files are absent changes nothing. Verify: expected · healthy is true and no error is present. If that does not apply: Attach the support bundle, which carries the log tail for every service.
check 4 fails	`dependency-unhealthy` (warning)	Some services report themselves unhealthy because something they need is down, not because they are broken. Fixing the wrong one wastes the maintenance window, and the dependency's own row already says so. Fix: Resolve the dependency first — the database for the MCP server, the MCP server and the local model for the analyst — then re-read the snapshot before touching this service. Verify: expected · The dependency is healthy and this service follows.

A container keeps restarting

Separate an out-of-memory kill from a configuration the service refuses, before restarting it again.

Runbookperformance

A container keeps restarting

You might see: has restarted 12 times — it is in a crash loop · container restarting · a service comes back and dies again · the dashboard works for a minute and then errors

Before you start

Has the appliance been up for more than about five minutes? Detection services take time to load their models on the first start.expected · At least five minutes have passed.
Which container is restarting?expected · You can name the service.If not, go to /troubleshoot/diagnostic-snapshot first.

Checks

1
Check whether the last termination was an out-of-memory kill rather than a crash
expected · oomKilled is false. When it is true, this is a sizing problem and not a crash.
2
Read how much memory the host has left
diagnostics · host.memFreeMb = 1024
3
Verify the schema is current, since a failing boot migration restarts the gateway forever
diagnostics · datastores.postgres.migrationsPending
4
Check whether the container refuses its own configuration at boot
expected · The service reports that it started and is listening, rather than refusing a setting.

Causes and fixes

Reached when	Cause	Explanation, fix and verification
check 1 fails	`repeated-oom` (critical)	The container is not crashing: the kernel is killing it, and the restart policy brings it back into the same conditions. Exit code 137 is the signature. Nothing in the service's own log explains it, because the process is stopped without warning. Fix: Follow A container was killed for running out of memory. Verify: expected · oomKilled is false and the restart count stops climbing.
check 2 fails	`host-memory-exhausted` (critical)	The host has almost no memory left, so the kernel kills whichever container is largest at the moment pressure peaks. The service that dies is not the service that is at fault, which is why the restarting container appears to move around between reboots. Fix: Give the machine more RAM, or reduce what runs on it. The full stack needs 32 GB, and 64 GB for a deployment that handles documents and images. See The machine is below the package's RAM minimum. Verify: diagnostics · host.memFreeMb = 1024
check 3 fails	`boot-migration-loop` (critical)	The gateway applies the migration chain at boot. A migration that throws takes the process with it, the container restarts, the same migration runs again, and the restart count climbs with nothing else changing. Fix: Stop the container, read one failed start end to end, and resolve the specific error. See Migrations are pending. Verify: diagnostics · datastores.postgres.migrationsPending
check 4 fails	`configuration-refused` (critical)	The service refuses to start with the configuration it was given, and says so in its first log lines. A placeholder signing secret is refused deliberately; a missing model path, an unreadable certificate and a bad database password behave the same way. The restart policy then turns a clear one-line refusal into a loop nobody reads. Fix: Fix what the log names, then start the container once and read the log again rather than leaving it to restart. If the refusal is about the signing secret, see The JWT signing secret is a placeholder. Verify: expected · The service reports that it is listening and the restart count stops climbing. If that does not apply: Attach the support bundle and the first twenty lines of one failed start.

A container was killed for running out of memory

Exit code 137 is the kernel, not the service. Restarting it restores service and changes nothing.

Runbookperformance

A container was killed for running out of memory

You might see: was killed for running out of memory (exit 137) · exit code 137 · an OOM kill (exit 137) means this machine needs more RAM · the detector dies under load

Before you start

Is the host agent reporting, so the memory numbers in the snapshot are real rather than absent?diagnostics · host.memTotalMb = 0If not, go to /troubleshoot/diagnostic-snapshot first.

Checks

1
Compare the host's total memory against the minimum for the licensed package
diagnostics · host.memTotalMb = 32768
2
Read how much memory the host has available right now
diagnostics · host.memFreeMb = 2048
3
Compare the killed container's declared limit with what it was using
expected · Usage sits well below the limit. Usage at the limit means the limit is the constraint, not the host.
4
Check whether every service running on this box is actually in use
expected · Everything running is something this deployment needs.

Causes and fixes

Reached when	Cause	Explanation, fix and verification
check 1 fails	`host-undersized` (critical)	The full stack was measured at about 30 GiB in use, and the default profile declares roughly 38 GB of container limits with several services declaring none. Below 32 GB the kernel reclaims memory by killing the largest process, which is always a detection service loading a model. The service that dies is a symptom of the machine, not a fault in itself. Fix: Raise the machine's memory to at least 32 GB, or to 64 GB where documents and images are processed, and reboot. See The machine is below the package's RAM minimum and Sizing and requirements. Verify: diagnostics · host.memTotalMb = 32768
check 2 fails	`no-headroom` (critical)	The machine meets the nominal minimum and has nothing left at this moment, so the next model load or document parse triggers a kill. Available memory, not total memory, is the number that predicts the next OOM. Fix: Reduce what runs concurrently, or add memory. Restarting the killed container restores service and does not change the outcome — the next peak kills something again. Verify: diagnostics · host.memFreeMb = 2048
check 3 fails	`limit-too-low` (warning)	The host has memory and the container hit its own declared limit, so the kernel killed it while the machine was fine. This is the honest case for raising a service's limit — it is also the only case, because raising a limit on a machine with no headroom just moves the kill to a different container. Fix: Raise that service's memory limit in the deployment's compose configuration and recreate the container. Confirm the host has the headroom for the new limit before you apply it. Verify: expected · oomKilled stays false and usage settles below the new limit.
check 4 fails	`unused-services-running` (info)	Several services are opt-in — OCR, document parsing, red-teaming, the local analyst and the MCP server — and each holds memory whether or not the deployment uses it. On an undersized box, switching off what is unused is faster than a hardware change and costs nothing that is in use. Fix: Stop the services this deployment does not use and confirm nothing depends on them. Document what you switched off: a later feature request will otherwise be diagnosed as a bug. Verify: diagnostics · host.memFreeMb = 2048

A PDF or report will not render

Rendering is a separate service. With it absent every PDF export fails and everything else on the page keeps working, which is why it looks specific to one button.

Runbookperformance

A PDF or report will not render

You might see: Failed to render compliance scorecard PDF · Failed to render detections PDF · Failed to render executive summary PDF · Failed to render regulations PDF

Before you start

Does the page you are exporting show data on screen?expected · The page shows the figures you expect in the PDF.If not, follow the dashboard-request-failed runbook instead.

Checks

1
Read whether the report renderer answered
diagnostics · services[report-renderer].reachable = true
2
Read whether the renderer reports itself healthy
diagnostics · services[report-renderer].healthy = true
3
Check whether only the large exports fail
expected · The small export succeeds.
4
For a compliance dossier, read whether the render was reconciled against the record
expected · The renderer reported its rendered ids.

Causes and fixes

Reached when	Cause	Explanation, fix and verification
check 1 fails	`renderer-not-reachable` (warning)	PDF rendering is a separate service the gateway calls over `REPORT_RENDERER_URL`. With that service absent or unreachable, every PDF export fails and everything else on the page keeps working — which is why the failure looks specific to one button. Fix: Start the report renderer and confirm the gateway's URL matches the port it listens on. Follow One service is unreachable or reports itself unhealthy if it will not come up. Verify: diagnostics · services[report-renderer].reachable = true
check 2 fails	`renderer-unhealthy` (warning)	A renderer that answers and reports itself unhealthy usually failed to initialise something it needs — a font set or a headless browser. It will keep answering and keep failing. Fix: Read the renderer's error and its container logs, then restart or rebuild the service. Verify: diagnostics · services[report-renderer].healthy = true
check 3 fails	`renderer-resource-limited` (warning)	Rendering holds the whole document in memory. A large date range or a wide estate produces a document that exceeds the container's memory limit, and the kernel kills the renderer rather than the request failing cleanly. Fix: Export a narrower range, or raise the renderer's memory limit. Check whether the container was OOM-killed — see A container was killed for running out of memory. Verify: expected · The PDF downloads.
check 4 fails	`dossier-unreconciled` (warning)	A compliance dossier is evidence. The product will not hand over a rendered document it could not check against the frozen record, because an unchecked report is worse than no report. The dossier JSON is unaffected and stays downloadable. Fix: Fix the renderer first, then re-render the dossier. Use the JSON in the meantime — it is the same record. Verify: expected · The PDF is produced and the download is allowed. If that does not apply: Attach the support bundle, the report and date range you exported, and the renderer's container log tail.

The LLM risk analyst is disabled, unreachable, or times out

The analyst and its MCP server are opt-in. When enabled they are the largest thing on the box and the first thing memory pressure kills.

Runbookperformance

The LLM risk analyst is disabled, unreachable, or times out

You might see: The risk analyst service is not enabled on this deployment. · The risk analyst could not be reached. · The analyst is already running an analysis. · The analysis did not start.

Before you start

Is this deployment supposed to run the local analyst at all?expected · The deployment is meant to run the analyst.If not, go to /administer/analyst first.

Checks

1
Read whether the gateway believes the analyst is enabled
expected · The flag is set.
2
Read whether the analyst service answered
diagnostics · services[risk-analyst].reachable = true
3
Read the MCP server, which the analyst reads the estate through
diagnostics · services[mcp-server].reachable = true
4
Check whether an analysis is already running
expected · No analysis is in flight.
5
For timeouts, compare the configured deadline against how long a reading actually takes
expected · The deadline is longer than a typical completed review took.

Causes and fixes

Reached when	Cause	Explanation, fix and verification
check 1 fails	`analyst-disabled` (info)	The analyst is off by default. Nothing runs per message behind the fast path — repeated detections fill the review window and open one per-user review, and with the analyst disabled those reviews have nowhere to go. Fix: Bring the stack up with the `llm-analyst` profile and set the enable flag, then restart the gateway. See Analyst. Verify: diagnostics · services[risk-analyst].reachable = true
check 2 fails	`analyst-not-reachable` (warning)	The analyst is a large local model service. It is the first thing memory pressure kills and the slowest thing to come back, so an appliance that is short of RAM shows this before it shows anything else. Fix: Follow One service is unreachable or reports itself unhealthy, and check for an out-of-memory kill before restarting it again. Verify: diagnostics · services[risk-analyst].reachable = true
check 3 fails	`mcp-down` (warning)	The analyst reads the estate through the read-only MCP server, which needs the database. With the MCP server down the analyst reports itself unhealthy even though nothing is wrong with the model — fixing the analyst wastes the window. Fix: Bring the MCP server back, and check PostgreSQL first — see PostgreSQL or Redis is unreachable. Verify: diagnostics · services[mcp-server].reachable = true
check 4 fails	`analyst-busy` (info)	One analysis at a time is a deliberate limit: the readings are long and running two would make both slower and neither finish sooner. Fix: Wait for the running analysis to finish, then start yours. Verify: expected · It is accepted.
check 5 fails	`deadline-too-short` (warning)	A timed-out review is recorded as timed out rather than silently retried. The fallback verdict then applies, and the shipped default is a warning rather than a lock — locking on silence is a false-positive landmine. Fix: Raise the timeout on LLM Review → Settings to match how long completed reviews take on this hardware, and check the fallback verdict is the one you want. Verify: expected · It completes rather than timing out. If that does not apply: Attach the support bundle, the review id, and the analyst's container log tail.

A dashboard action fails with "Failed to …" and no reason

The dashboard's failure toasts are generic by design. The HTTP status separates five unrelated causes in one step.

Runbookperformance

A dashboard action fails with "Failed to …" and no reason

You might see: Failed to create · Failed to update · Failed to delete · Error loading data. Please try again.

Before you start

Does one action fail, or does the whole page fail to load?expected · You can say which of the two it is.If not, follow the first-boot-no-dashboard runbook instead.

Checks

1
Read the HTTP status the request actually returned
expected · You have a status code and a response body.
2
For 401, check whether the session simply expired
expected · The action succeeds after signing in again.
3
For 402, 403 or 423, read the error code
expected · The request is not being refused by role, licence or setup state.
4
For 400 or 409, read the message the server sent
expected · The message does not describe something you can correct in the form.
5
For 5xx or no response at all, check that the gateway is serving
diagnostics · services[nestjs-core].healthy = true

Causes and fixes

Reached when	Cause	Explanation, fix and verification
check 1 fails	`no-status-available` (info)	The dashboard's failure toasts are generic by design — they are written once and reused across every resource. Diagnosing from the toast alone is guessing; the status code separates five unrelated causes in one step. Fix: Reproduce the action with the network panel open, or find the request in Event Logs. Verify: expected · You can read the status and body.
check 2 fails	`session-expired` (info)	Tokens expire (24 hours by default), and a tab left open overnight keeps rendering the page it already loaded while every new request is refused. The page looks healthy and every action fails. Fix: Sign in again. If sessions end at every gateway restart, follow The token signing secret is a placeholder. Verify: expected · It succeeds.
check 3 fails	`refused-by-policy` (info)	Role, licence and setup refusals are deliberate and carry a code. The dashboard renders them through the same generic toast as a transport failure, which is what makes them look like faults. Fix: Follow The product refuses an action with 402, 403 or 423. Verify: expected · It succeeds.
check 4 fails	`request-rejected` (info)	A 400 is the server telling you what to change; a 409 usually means someone else changed the same row while your screen was open, and the page must be reloaded before saving again. Fix: Correct the field the message names, or reload the screen and reapply your change on top of theirs. Verify: expected · It succeeds.
check 5 fails	`gateway-unhealthy` (critical)	A 5xx or a request that never answers is the gateway, not the dashboard. One dependency being down — the database, a detection service, the renderer — takes out only the endpoints that need it, so most of the dashboard keeps working. Fix: Read the findings in the diagnostic snapshot and follow the runbook the finding names. Start with PostgreSQL or Redis is unreachable and Migrations are pending, which produce 500s on a subset of endpoints while everything else looks healthy. Verify: diagnostics · services[nestjs-core].healthy = true If that does not apply: Attach the support bundle, the exact route and status, and the time of the failure so the gateway log can be correlated.

Inline proxy and inspection

Every ICAP caller arrives as the same docker bridge address

A gateway running as a virtual machine beside CID222 does not reach the listener with its own address, so a source-IP allowlist there admits all callers or none.

Runbookinline-proxy

Every ICAP caller arrives as the same docker bridge address

You might see: every SWG appears as 172.19.0.1 · the ICAP source-IP allowlist does not distinguish our proxies · allowed_source_ips has no effect · the allowlist blocked every caller after we set it

Before you start

Is the secure web gateway a virtual machine on the same host as CID222, rather than a physical appliance on the LAN?expected · You can say which. The answer decides whether a source-IP control can work at all.
Is inline-proxy inspection actually deployed here, rather than the API gateway path alone?expected · Inspection is deployed.If not, go to /deploy/inline-proxy first.

Checks

1
Read how the ICAP listener admits callers
diagnostics · posture.inspectionSourceIpMode = "unrestricted"
2
Check whether callers arrive with their own addresses or with the container bridge address
expected · The recorded source address is the gateway's own address.
3
When an explicit allowlist is set, verify it lists every caller that must be admitted
expected · Every caller is listed.

Causes and fixes

Reached when	Cause	Explanation, fix and verification
check 2 fails	`nat-rewrites-the-source` (warning)	A gateway running as a virtual machine beside CID222 does not reach the listener with its own address. The VM network's own masquerade rule rewrites the source once the destination leaves that subnet, so both a multipass and a libvirt guest were measured arriving as the same container bridge address. This is a property of same-host laboratories: a gateway on a physical LAN keeps its address and the control works. Fix: Do not rely on a source-IP allowlist in a same-host VM laboratory — it cannot distinguish the callers, and setting it either admits all of them or none. To test an allowlist in that environment, add a rule that returns the container subnet's traffic ahead of the masquerade rule. In production, place the gateway on the network rather than on the appliance host. Verify: expected · The address is the gateway's own, not the container bridge address.
check 1 fails	`unrestricted-admission` (warning)	The listener cannot authenticate its peer — no product it integrates with offers a credential or a custom header — so admission is by address alone, and an admitted peer is believed down to the user identity it claims in each request. Unrestricted admission means anything that can reach the port can assert any user. Fix: Restrict the listener to the addresses of the gateways that legitimately call it, and place the port on a segment those gateways alone can reach. Treat network placement, not the allowlist, as the real control. Verify: diagnostics · posture.inspectionSourceIpMode = "unrestricted"
check 3 fails	`allowlist-replaced-the-default` (critical)	An explicit allowlist replaces the private-range default rather than extending it. Setting one address therefore removes admission for every other caller, including ones that worked a moment earlier, and the failure appears at the gateway as a refusal rather than at the listener as a configuration change. Fix: List every caller that must be admitted, not only the one you were adding, then re-test each gateway. Values written before this control began to be enforced were cleared during the upgrade and recorded in the event log, so a value you remember setting may no longer be in place. Verify: expected · Every one is admitted.

The inspection endpoint refuses the proxy calling it

Admission is evaluated in a fixed order — an mTLS requirement, then a bearer key, then network position — and a key that resolves to no tenant is refused outright rather than treated as no key at all.

Runbookinline-proxy

The inspection endpoint refuses the proxy or gateway calling it

You might see: Client certificate required · API key required · Invalid API key · Internal API only

Before you start

Is an inline proxy, SWG or ICAP client actually configured to call this appliance?expected · A caller is configured.If not, go to /deploy/inline-proxy first.

Checks

1
Read whether this deployment mandates a client certificate
expected · Either no DN is required, or the caller presents a certificate matching it.
2
Read whether the caller presents a gateway API key
expected · The caller sends `Bearer cid_key_…` and the key resolves.
3
Check whether the call arrives through a reverse proxy
expected · Either the caller presents a key, or it reaches the listener directly from an internal address.
4
Before trusting any source-IP control, check what address the caller actually arrives as
expected · Callers arrive with their own addresses.

Causes and fixes

Reached when	Cause	Explanation, fix and verification
check 1 fails	`mtls-required-not-presented` (critical)	An mTLS requirement is an additional constraint, evaluated first. It was written, displayed and read by nothing until 2026-08-29; because a value set before then was never applied, the upgrade clears it and records the previous value in the event log. A value re-entered after the upgrade does enforce. Fix: Present a client certificate whose subject DN matches the configured value, or clear the DN if this deployment does not run mTLS. Re-read the event log for the value the upgrade cleared before re-entering one. Verify: expected · It is accepted.
check 2 fails	`key-missing-or-unresolvable` (critical)	Returning "no tenant" for a key that resolves to nothing would be indistinguishable from a call with no key, and callers fall back to the tenant named in the body in that case — which would let a key holder attribute records to any tenant. So the key is refused instead. Fix: Issue a gateway API key for the caller and configure it as a bearer token. Follow A gateway API key is refused when the key exists and is still refused. Verify: expected · It is accepted and the resulting records carry the expected tenant.
check 3 fails	`keyless-through-edge` (warning)	Behind a reverse proxy every caller looks internal, so the private-network allowance would admit the whole internet. The listener refuses keyless calls that came through an edge and says so. Fix: Give the caller a gateway API key, or have it reach the listener directly. Verify: expected · It is accepted.
check 4 fails	`source-ip-is-the-bridge` (warning)	A SWG running as a VM beside CID222 arrives at the listener as the docker bridge address, because the source network's own NAT masquerades it. A source-IP allowlist in that situation admits every caller or none, and proves nothing either way. On a physical LAN the address survives and the allowlist works. Fix: Follow Every ICAP caller arrives as the same docker bridge address. Verify: expected · The caller arrives with its own address. If that does not apply: Attach the support bundle and the listener's rejection log lines — they name the apparent source address and the reason.

The proxy CA or a helper download is not available

Recreating the proxy with a new volume generates a new CA. Every client that trusted the old one must be given the new one; old trust does not transfer.

Runbookinline-proxy

The inline proxy's CA or helper download is not available

You might see: CA not available. Is cid-inline-proxy running? · Unsupported format · dc-agent.ps1 not bundled in this build · the CA download button returns an error

Before you start

Was the stack started with the inline-proxy profile?expected · The container exists.If not, go to /deploy/inline-proxy first.

Checks

1
Read the container state for the inline proxy
expected · state is running.
2
Check whether the proxy has generated its CA yet
expected · The proxy has completed a first start and holds a CA.
3
For an "Unsupported format" refusal, read which format was asked for
expected · The format is one of the offered ones.
4
For the domain-controller helper script, check whether this build ships it
expected · The build carries the helper.

Causes and fixes

Reached when	Cause	Explanation, fix and verification
check 1 fails	`proxy-not-running` (critical)	The CA is served from the proxy's own state. With the container down there is nothing to read, and every client that has not already trusted the CA sees a certificate warning on each intercepted site. Fix: Start the container and wait for its healthcheck, then retry the download. See A container keeps restarting when it will not stay up. Verify: expected · The file downloads.
check 2 fails	`ca-not-generated` (warning)	Recreating the proxy with a new volume discards the CA and generates another one. Every client that trusted the old CA must be given the new one — the old trust does not transfer. Fix: Let the proxy start cleanly, download the new CA, and redistribute it to the clients. Plan this as a change, not as a restart. Verify: expected · No certificate warning.
check 3 fails	`format-not-offered` (info)	The download endpoint serves a fixed set of formats. An unknown one is refused rather than guessed at, because a mislabelled certificate file fails silently on the client. Fix: Request one of the formats the page offers. Verify: expected · The file downloads.
check 4 fails	`helper-not-bundled` (info)	The helper script is a source-tree artefact, not part of every image. Its absence is a build property, not a fault on this appliance. Fix: Take `dc-agent.ps1` from `cid-userid-agent/examples/` in the source tree, as the message says. See User-ID attribution. Verify: expected · It runs on the domain controller. If that does not apply: Attach the support bundle and the build version shown on the System Updates page.

Endpoint agent and browser extension

An endpoint agent will not enrol, or stops receiving policy

A fleet that is not active refuses new enrolments with the same message as an unknown token, and an agent adopts a bundle only when its version is strictly higher than the one it applied.

Runbookendpoint-agent

An endpoint agent will not enrol, or stops receiving policy

You might see: Invalid enrollment token · Missing agent API key · Invalid agent API key · Agent key does not match device

Before you start

Does the fleet the agent is enrolling into exist and is it active?expected · The fleet exists and is active.If not, go to /administer/endpoint-agent first.

Checks

1
Check the enrollment token against the fleet
expected · The token belongs to a fleet whose status is active.
2
Read the Authorization header the agent sends on policy and heartbeat calls
expected · The agent sends a key the gateway recognises.
3
Check that the key and the device id agree
expected · The key resolves to the device id the agent claims.
4
Read the fleet's policy for the two fields the bundle cannot be signed without
expected · Both are present.
5
For a policy change the fleet never adopted, check that the bundle version advanced
expected · The version is higher than the one the agent reports.

Causes and fixes

Reached when	Cause	Explanation, fix and verification
check 1 fails	`token-or-fleet-inactive` (warning)	Enrolment matches the token's hash to a fleet and requires that fleet to be active. Deactivating a fleet therefore stops new enrolments without revoking the devices already in it, and both failures read as an invalid token. Fix: Re-activate the fleet, or issue the agent the token of a fleet that is active. Verify: expected · The device appears in the fleet.
check 2 fails	`agent-key-missing-or-invalid` (critical)	After enrolment the agent authenticates with a gateway API key like any other API client, so the key's own lifecycle applies: revoke it or let it expire and the agent goes quiet without any endpoint-side change. Fix: Check the key under Access Keys and follow A gateway API key is refused. Re-enrol the device if the key cannot be restored. Verify: expected · It is recent.
check 3 fails	`key-device-mismatch` (warning)	A key is bound to one device id. Cloning a machine image after enrolment is the usual cause: two machines then present the same key with different device ids, and both are refused. Fix: Re-enrol the affected machines so each gets its own key, and exclude the agent's state from the golden image. Verify: expected · Each machine appears once, with a recent heartbeat.
check 4 fails	`policy-incomplete` (warning)	The signed bundle must state how selective the agent is and what it does when it cannot reach the appliance. A bundle missing either is refused at signing rather than shipped with a default the operator never chose. Fix: Set `selectivity` and `failMode` on the fleet policy and save it again. Verify: expected · A signed bundle is returned.
check 5 fails	`version-did-not-move` (info)	Agents adopt strictly-increasing bundle versions, so a change that does not advance the number is a change the fleet never hears about. Lock and unlock transitions move the number through a generation offset for exactly this reason. Fix: Save the policy again so its version advances, then wait one poll interval. Verify: expected · It matches the fleet's current version. If that does not apply: Attach the support bundle, the fleet id, the device id and the agent's own log.

A device never enrolled, a device enrolled to another tenant, key material of the wrong shape and a device throttled for re-attesting in a loop are four different failures.

Runbookextension

The browser extension cannot attest, sign in, or fetch its policy

You might see: Attestation rejected · Device is not enrolled · Device is enrolled to another tenant · Device not found

Before you start

Is the extension configured to point at this appliance, over a name the browser trusts?expected · The endpoint is this appliance and its certificate validates in the browser.If not, follow the tls-expiry-or-443-not-served runbook instead.

Checks

1
Read the device's enrolment
expected · The device is enrolled to the tenant the user belongs to.
2
For a rejected enrolment, read what key material the extension sent
expected · The extension sends a P-256 public JWK.
3
Check whether the device is being refused for frequency rather than identity
expected · The device attests at its normal interval.
4
Read the role of the account signing in
expected · The role is one that uses the extension.
5
For the firewall EDL feed, read the token the firewall presents
expected · The firewall presents a current EDL token.

Causes and fixes

Reached when	Cause	Explanation, fix and verification
check 1 fails	`device-enrolment-wrong` (warning)	A device belongs to exactly one tenant. Re-imaging a machine, or moving a user between tenants without re-enrolling, leaves the device attesting against a tenant that no longer matches the signed-in user. Fix: Remove the stale device record and enrol the machine again as the current user. See Browser extension. Verify: expected · The device reports enrolled and healthy.
check 2 fails	`bad-key-material` (info)	Attestation is a signature check. A key of the wrong curve, or one that carries a private scalar, cannot be used for it and is refused rather than stored. Fix: Reinstall or reset the extension so it generates a fresh key pair, then enrol again. Do not copy key material between machines. Verify: expected · Enrolment succeeds.
check 3 fails	`attestation-throttled` (info)	The attestation endpoint is rate limited per device. A device that crashes and re-attests in a loop is refused for frequency, which masks whatever is actually crashing it. Fix: Stop the loop first — read the extension's own error — then let the throttle window pass. See A request is refused for being too frequent. Verify: expected · The attestation is accepted.
check 4 fails	`role-has-no-extension-view` (info)	The refusal is deliberate and is not a fault: the account authenticated successfully and its role has no extension surface. It is the same design as the chat refusal for audit roles. Fix: Use the dashboard with that account, or give the user an account whose role uses the extension. Verify: expected · The extension opens.
check 5 fails	`edl-token-missing-or-expired` (warning)	The EDL feed is consumed by a firewall that cannot sign in interactively, so it carries its own token. An expired token stops the feed updating; the firewall keeps enforcing the last list it fetched, which makes the failure quiet. Fix: Issue a new EDL token in the dashboard and update the firewall's feed URL. Check the feed after the change rather than assuming it recovered. Verify: expected · The feed returns the current list. If that does not apply: Attach the support bundle, the device id and the extension's own log.

Integrations and repositories

Events stop arriving at the SIEM or webhook collector

Nothing is dropped silently — the exporter advances its cursor only after a successful send — so a stalled export has a reason attached and a zero drop count is structural rather than reassuring.

Runbookdata

Events stop arriving at the SIEM or webhook collector

You might see: a backlog stopped moving · the collector kept refusing delivery · Configuration store unavailable; settings cannot be saved · ITSM handoff is unavailable in this process; cannot send a test event

Before you start

Is an outbound integration configured and enabled?expected · It exists and is enabled.If not, go to /deploy/siem first.

Checks

1
Read why the exporter stopped
expected · A reason is recorded.
2
Send a test event to the collector
expected · The collector accepts it.
3
For syslog, check host, port and protocol
expected · The collector is reachable on the configured transport.
4
For "settings cannot be saved", check the configuration store
diagnostics · datastores.postgres.reachable = true

Causes and fixes

Reached when	Cause	Explanation, fix and verification
check 1 fails	`no-reason-recorded` (info)	Nothing is dropped silently: the exporter advances its cursor only after a successful send, so a failed batch is retried rather than skipped. A stalled export therefore has a reason attached, and a zero drop count is structural rather than reassuring. Fix: Read the integration's status panel and the event log entries for the exporter. Verify: expected · A reason or a moving cursor is visible.
check 2 fails	`collector-refusing` (warning)	A collector that refuses delivery keeps the backlog in place rather than losing it, so the symptom is an export that stops advancing rather than events disappearing. Refusals are usually authentication, a payload the collector rejects, or a header the gateway sets that the collector will not accept. Fix: Read the collector's own logs for the refusal. Note that some headers are set by the gateway and cannot be overridden, and custom headers are capped in length. Verify: expected · The collector accepts it and the backlog starts moving.
check 3 fails	`transport-blocked` (warning)	Syslog over UDP is fire-and-forget: the appliance cannot tell whether anything received it, so a blocked path looks exactly like a working one until you check the collector. TCP and TLS fail visibly instead. Fix: Open the path, or move the integration to a transport that reports failure. Confirm at the collector, not at the appliance. Verify: expected · It arrives.
check 4 fails	`config-store-unavailable` (critical)	Integration settings live in the configuration store. With the database unreachable the settings cannot be written, and the refusal is explicit rather than a save that appears to work and is lost. Fix: Follow PostgreSQL or Redis is unreachable. Verify: diagnostics · datastores.postgres.reachable = true If that does not apply: Attach the support bundle, the integration type and the collector's own log lines for the window in which delivery stopped.

A repository connector cannot read the repository

Credentials are encrypted and cannot be read back, so a connector saved with a missing field looks complete on screen and fails at first use.

Runbookdata

A repository connector cannot read the repository

You might see: generic_git connector needs base_url set to the clone URL · GitHub App connector needs both app_id and private_key_pem · connector needs an access_token · Assignment has no GitHub installation id

Before you start

Is repository tracking switched on for this deployment?expected · The flag is set.If not, go to /administer/repo-connector first.

Checks

1
Read the connector's credential fields for its type
expected · Every field the type needs is set.
2
Read the clone URL
expected · The URL is one of the accepted forms.
3
Check that the credential type and the URL scheme can work together
expected · The URL scheme matches the credential the connector holds.
4
For GitHub, read the host URL field
expected · Blank for github.com, or the bare GHES host.
5
Before reading findings, check that a baseline scan completed
expected · A completed baseline exists for the repository.
6
Read whether the last scan ran with every analyzer available
expected · The scan did not run degraded.

Causes and fixes

Reached when	Cause	Explanation, fix and verification
check 1 fails	`connector-incomplete` (info)	Credentials are encrypted at rest and cannot be read back, so a connector saved with a missing field looks complete on screen and fails at first use. The refusal names the field. Fix: Re-enter the missing field and save the connector again. Verify: expected · It starts.
check 2 fails	`clone-url-shape` (info)	The URL is validated before it is handed to git, including the leading-hyphen case — a URL beginning with a hyphen would be parsed as a command-line option by git itself. Fix: Correct the clone URL to one of the accepted forms. Verify: expected · It saves.
check 3 fails	`auth-method-mismatch` (warning)	Deploy keys are an ssh mechanism. Pairing one with an https clone URL cannot work, and the product says so at configuration time rather than failing on every scan afterwards. Fix: Use an ssh clone URL with a deploy key, or a token with an https URL. Verify: expected · The repository is read.
check 4 fails	`host-url-wrong` (info)	A repository URL in the host field points the connector's API calls at a path that is not an API root, and every call fails in a way that looks like a permissions problem. Fix: Leave it blank for github.com, or set it to the GHES host alone. Verify: expected · Repositories are listed.
check 5 fails	`no-baseline` (info)	Incremental tracking is a diff against a baseline. Until one completes there is nothing to diff against and nothing to read files at, so findings and analyses are empty rather than wrong. Fix: Run a baseline scan and let it finish. Check the event log for a scan that started and did not complete. Verify: expected · A completed baseline is listed.
check 6 fails	`scan-degraded` (warning)	A degraded scan ran with an analyzer unavailable. Its result must be read as unknown, not as clean — that distinction is the whole point of recording the degradation. Fix: Bring the missing analyzer back — see One service is unreachable or reports itself unhealthy — and re-run the scan before acting on its result. Verify: expected · It completes without the degraded marker. If that does not apply: Attach the support bundle, the connector type and the repository's scan history.

If none of these match

Read the diagnostic snapshot end to end. Its findings section names a runbook for each rule that fired.
Check the health matrix — a service that is reachable but unhealthy is the air-gapped failure mode.
Collect the support bundle and escalate. Include the installation id and the appliance version.

Guided runbooks

Before you start

Checks

Causes and fixes

Before you start

Checks

Causes and fixes

Before you start

Checks

Causes and fixes

Before you start

Checks

Causes and fixes

Before you start

Checks

Causes and fixes

Before you start

Checks

Causes and fixes

Before you start

Checks

Causes and fixes

Before you start

Checks

Causes and fixes

Before you start

Checks

Causes and fixes

Before you start

Checks

Causes and fixes

Before you start

Checks

Causes and fixes

Before you start

Checks

Causes and fixes

Before you start

Checks

Causes and fixes

Before you start

Checks

Causes and fixes

Before you start

Checks

Causes and fixes

Before you start

Checks

Causes and fixes

Before you start

Checks

Causes and fixes

Checks

Causes and fixes

Before you start

Checks

Causes and fixes

Before you start

Checks

Causes and fixes

Before you start

Checks

Causes and fixes

Before you start

Checks

Causes and fixes

Before you start

Checks

Causes and fixes

Before you start

Checks

Causes and fixes

Before you start

Checks

Causes and fixes

Before you start

Checks

Causes and fixes

Before you start

Checks