# Upgrade and roll back

## Before you upgrade

1. Read the release notes for schema or protocol changes.
2. Confirm you hold `NEBULACTRL_SECRET_KEY` ([backup and restore](https://nebulactrl.com/docs/operations/backup-restore.md)). The backup of the database is taken **for you**: the installer backs the control plane up before it changes anything, and the control plane does the same at startup when its version changed ([how it works](https://nebulactrl.com/docs/operations/upgrades.md#backup-before-a-control-plane-upgrade)). **If that backup fails the upgrade stops**; fix the cause, or re-run with `--force-upgrade` to go on without one.
3. Note the version you run now (`docker compose -f /opt/nebulactrl/docker-compose.yml --env-file /opt/nebulactrl/.env images`).

## Upgrade the control plane (and the host's agent)

Re-run the installer with the new version. Nothing that exists is regenerated:

```sh
curl -fsSL https://nebula.example.com/install.sh | sudo bash -s -- --version 1.4.0
# or, with an image you built or saved yourself:
sudo bash install.sh --image nebulactrl:1.4.0 --image-archive nebulactrl-1.4.0.tar
```

The installer keeps `.env` and the database, loads or pulls the image, recreates only the containers whose definition changed, waits for `/readyz`, and upgrades the host's agent to the binary that ships with the new control plane. Your options are remembered in `/opt/nebulactrl/install.conf`.

- The server applies **forward-only** database migrations on startup, under a Postgres advisory lock. There is no automatic down migration.
- Startup order is safe: Postgres is kept, the server container is replaced, agents reconnect on their own (exponential backoff, up to 60 s) and the control plane **never stops remote containers** while it is down. Your services keep serving through a control-plane restart or upgrade, including those on the control-plane host (the proxy is the agent's, not the control plane's).
- Commands acknowledged before the restart keep their state; the agent redelivers unacknowledged results after reconnecting.
- The upgraded control plane serves new agent binaries immediately, and **agents upgrade themselves** when they reconnect (below).

## Agents upgrade themselves

An agent that is behind the control plane downloads the new binary from its control plane, verifies the SHA-256, replaces itself atomically, restarts when its running work is done and rolls back if the new binary does not start ([details](https://nebulactrl.com/docs/operations/upgrades.md)). This is automatic by default (Servers page → **Upgrade automatically**); **Upgrade agents** does the same one server at a time on request, and each server's page shows the version skew. Your services keep running.

## Upgrade other servers by hand

You need this once for an agent that predates self-upgrade, for a server that is offline during the upgrade, and when you switched the automatic upgrade off. Re-run the agent installer on the host (no token needed; the identity is kept):

```sh
curl -fsSL https://nebula.example.com/install-agent.sh | sudo sh -s -- --url https://nebula.example.com
```

It downloads the new binary, verifies it against the checksum the control plane publishes, restarts the service, and verifies the service stays up. Managed containers and the Traefik proxy keep running during the agent restart. The wire protocol is versioned; a control plane refuses an unknown **major** protocol version with close code 4400, so upgrade the control plane first and agents second. The server card shows each agent's version.

## Roll back the control plane

Because migrations only go forward, an older server binary may not work against a newer schema. The supported rollback is:

1. `cd /opt/nebulactrl && docker compose down` (do **not** pass `-v`).
2. Restore the pre-upgrade backup (Settings → Instance → History, trigger *Before upgrade*; or `/var/backups/nebulactrl/pre-upgrade-*.dump` from the installer) into a clean database ([restore](https://nebulactrl.com/docs/operations/backup-restore.md#restore)). Changes made after the dump (deployments, variable edits) are lost.
3. Run the installer with the previous `--version` (or image).

If the new version only changed code (no new migration, see `db/migrations` in the diff), re-running the installer with the previous version is enough.

## Roll back an agent

A failed self-upgrade rolls back by itself. After a good one the previous binary stays at `/usr/local/bin/nebulactrl-agent.prev`: move it over the binary and `systemctl restart nebulactrl-agent`. Or re-run the installer from an older control plane, or place an older binary at `/usr/local/bin/nebulactrl-agent`. The control plane never downgrades an agent, and an agent newer than the control plane is not upgraded. Use an agent whose protocol major version the control plane accepts.

## Roll back an application (not the platform)

That is a deployment rollback inside the UI/API/CLI: it re-deploys an earlier release's image and configuration. It does **not** restore data. See [operations: deployments](https://nebulactrl.com/docs/operations/deployments.md).