Troubleshooting

Start with: which part is failing, the control plane (docker compose logs nebulactrl), the agent (journalctl -u nebulactrl-agent), or the proxy (docker logs nebulactrl-traefik on the agent host)?

The agent does not connect#

Proxy: port 80 or 443 in use#

The server card shows Proxy error and adding a domain fails with 422 proxy_unavailable. Find the occupant:

sudo ss -ltnp 'sport = :80 or sport = :443'
docker logs nebulactrl-traefik 2>&1 | tail -20

Typical culprits: a host nginx/Apache, or another reverse-proxy container. (The control plane's own compose file has no proxy: the host's agent's Traefik serves the dashboard.) Stop it, or move NebulaCtrl's proxy: sudo systemctl edit nebulactrl-agent, add --proxy-http-port / --proxy-https-port to ExecStart (or put NEBULACTRL_PROXY_HTTP_PORT=8080 into /etc/nebulactrl-agent.env) and restart. Public HTTPS certificate issuance needs the real 80/443 reachable from the internet.

The proxy also needs 127.0.0.1:9000 for its admin entrypoint. If something else (Portainer, for example) uses that port, the proxy fails to start the same way; move it with --proxy-admin-port (or NEBULACTRL_PROXY_ADMIN_PORT).

The agent creates nebulactrl-traefik when it starts, so after freeing the port run sudo systemctl restart nebulactrl-agent; the heartbeat then reports the proxy as running.

A domain does not work#

1. DNS: the domain row shows DNS status. A records must point at the server's public IP; a job rechecks every 10 minutes (dns_ok is empty when the server has no public IP configured). Wildcard CNAMEs to a proxy such as Cloudflare in orange-cloud mode make the check say "mismatch" although it may work.
2. Certificate: Traefik needs port 80 reachable from the internet for the ACME HTTP-01 challenge and an ACME email on the server; only non-local names get Let's Encrypt certificates. Look at docker logs nebulactrl-traefik 2>&1 | grep -i acme. Let's Encrypt rate limits apply to repeated failures; wait or use another hostname.
3. 502 / "no upstream": the service is not healthy or not on the edge network. docker ps --filter name=nc-<slug>; check the deployment succeeded.
4. Hostname conflict: 409 domain_conflict: a hostname belongs to exactly one service in the whole installation.
5. Local testing: *.localhost, *.local, bare names and IPs get Traefik's default self-signed certificate: use curl -k. Unknown hosts get Traefik's default 404.

A deployment fails#

Open the deployment: the step timeline says where, the log says why.

Disk full on an agent host#

Images, the build cache and volumes share Docker's data root. Check the disk gauge (warns at the per-server threshold, default 80 %). The server's Docker cleanup runs daily and starts by itself when the alert is active; press Run cleanup now on the server page to reclaim at once, or lower a service's image retention. Never docker volume prune blindly: it removes service data (the cleanup's Remove unused volumes keeps every volume of your services).

The UI looks stale#

If the pill "Live updates paused — retrying" is visible, the event stream (/api/v1/events, Server-Sent Events) is blocked, usually by a proxy that buffers or has a short timeout. Data still refreshes every 15 s. Disable buffering for text/event-stream in your reverse proxy.

Control plane will not start#

Locked out#

Lost the password of the only Owner, or the phone with the authenticator app? Both are fixed from the control-plane host; nothing needs SMTP or another admin. Run the commands with the server's environment, in the standard install (/opt/nebulactrl; the Compose service is nebulactrl, or control-plane in deploy/compose):

cd /opt/nebulactrl
docker compose exec nebulactrl nebulactrl-server reset-password you@example.com
docker compose exec nebulactrl nebulactrl-server reset-2fa you@example.com

• reset-password <email> prints a one-time link (<dashboard>/reset-password?token=ncr_…, works once, valid 30 minutes). Open it and choose a new password. If NEBULACTRL_PUBLIC_URL is not set the output shows the path only: put the address you open the dashboard at in front of it. --generate sets a random password instead and prints it once; use it for an account that has no password (passkey or single sign-on only). It asks you to type the account email first (--confirm <email> answers in advance).
• reset-2fa <email> removes the authenticator app (also a half-finished enrolment and its lockout), every passkey and every recovery code, after you type the account email (--confirm <email>). Afterwards the account signs in with its password alone until it enrols again in Settings → Security; a team that requires 2FA asks for that at the next sign-in. It refuses for an account with no password and no linked provider whose only way in is its passkeys: run reset-password --generate first.
• A lost password and a lost second factor: run both. A reset link never bypasses 2FA.
• Both end every browser session of the account and are audited as system actions (operator.password_reset_link, operator.password_set, operator.2fa_reset, in each of the account's teams, with the OS user that ran them; never the link or the password). API tokens are not revoked: revoke them in Settings → Tokens if you suspect a leak.
• No accounts yet (error: this instance has no accounts yet…): first-run setup is still pending. Open the setup URL the installer printed (<dashboard>/setup?token=…, also in /opt/nebulactrl/INSTALL-SUMMARY.txt).
• Whoever can run these has the database and the secret key already; see login methods for the by-email flow and the team-admin reset.

Reporting a problem#

Include the NebulaCtrl version, the deployment ID, the relevant correlation_id log lines, and journalctl -u nebulactrl-agent around the time. Remove secrets first. Security problems: see SECURITY.md.