Upgrade

Move a running Warden to the current release in one step, with an automatic way back if the new version fails its check.

What this is

install.sh --upgrade is the supported way to upgrade Warden. It replaces the binary and the systemd units, and on a 0.41.x host it also migrates your configuration from schema 4 to schema 5. It does all of this as one transaction: if a step fails, it puts the old version back.

This page covers hosts that already run Warden 0.41.x, 0.42.0, 0.43.0 or 0.44.0. The 0.44.5 installer upgrades each of them in one step. What the upgrade does shows what happens to the configuration of each. For a first install, see the quick start.

0.42.0 and 0.43.0 were pre-releases. The stable releases before 0.44.5 were 0.41.0 and 0.44.0, so most hosts upgrade from 0.44.0, or straight from 0.41.x. From 0.41.x, everything that changed in 0.42.0, 0.43.0 and 0.44.0 is new to you as well. Before you start, read the notes of every release after the one you run, on the releases page.

Before you upgrade

• The daemon must be running. A 0.41.x host needs a running daemon. A 0.42.0, 0.43.0 or 0.44.0 host may also be in the failed state; see Upgrading a host whose daemon will not start.
• Wildcard, regex and $important rules (from 0.41). Schema 5 reads some of these rules differently. If the check finds one, the upgrade stops with nothing changed and prints the command that lists the findings. See When the check finds something to review.
• IP blocklists. If [ip_blocklists] is enabled, every source must be a public https:// URL, with no user name or password and no private or loopback address. 0.41 skipped such sources with a warning, so they filtered nothing. warden config lint and the upgrade check now refuse them, and the upgrade stops with nothing changed. Fix or remove each such source first.
• Backups made by 0.41. The new version restores schema 5 archives only. It refuses an archive from 0.41 or earlier, for example with schema-4 archive (Warden 0.41 or earlier); live config untouched. If you may need one of those archives, restore it before you upgrade. Take a fresh warden config backup once the upgrade is done.
• Disk space. Each policy write keeps undo data in the configuration directory for at least 24 hours. That data is capped at 2 GiB per configuration. A write that would pass the cap is refused with UndoStoreFull until older data expires.
• A cluster secondary on 0.41 cannot be upgraded in place. See Cluster secondaries on 0.41.
• A host that resolves names through its own Warden. While the daemon is stopped, such a host resolves nothing, so the bundle download and the installer’s network check fail. This matters only when the daemon is already down (see the failed-daemon case). In that case, add a temporary nameserver line to /etc/resolv.conf, and remove it afterwards.

Run the upgrade

Fetch the installer fresh and run its upgrade mode:

$ curl -fsSL https://get.purge.cc | sudo sh -s -- --upgrade

A host without curl, such as a Debian host installed with wget, runs wget -qO- https://get.purge.cc | sudo sh -s -- --upgrade instead. The curl command would print curl: command not found, change nothing, and still exit with status 0.

Add --yes after --upgrade to skip the confirmation prompt.

• Do not reuse an old copy of get.sh. A copy saved from 0.41.0 or 0.42.0 refuses the current bundle with the bundle contains an unexpected path. It installs nothing. Always fetch the script again.

• Preview first. Add --dry-run to see what the upgrade would do. The preview changes nothing:

  bash copy

  curl -fsSL https://get.purge.cc | sudo sh -s -- --upgrade --dry-run
    

• Without --upgrade the installer refuses a host that already has Warden and tells you to rerun it with the flag.

• From an unpacked bundle, run this in the bundle directory:

  bash copy

  sudo ./scripts/install.sh --upgrade --binary ./warden
    

• From a source checkout, build first, then upgrade:

  bash copy

  cargo build --release
  sudo ./scripts/install.sh --upgrade
    

  make install is for a first install only and skips the configuration check. Use install.sh --upgrade for an existing host.

• On aarch64, get.purge.cc has no bundle. get.sh stops with download failed and points at building from source. Build as described in Build from source, then run sudo ./scripts/install.sh --upgrade --binary <path> from the checkout, or sudo ./scripts/install.sh --upgrade --build-from-source.

The installer writes a log of every real run to /var/log/purge-warden-install-<date>-<time>.log.

What the upgrade does

The upgrade runs in this order:

1. It checks your configuration with the new binary, before it changes anything. An unknown or unreadable schema is refused with nothing changed.
2. It stops the daemon and the scheduled backup, and keeps the old binary, the old service unit and a copy of the publication store.
3. It plans and applies the migration for your schema (see the table below).
4. It installs the new binary and the service unit, and starts the daemon.
5. It checks that the new daemon answers and blocks. Only then does it finalize the migration.
6. It finishes the post-upgrade steps: the admin token, the backup directory, the backup timer units and the warden wrapper.

DNS answers stop at step 2, when the old daemon stops. They resume only when the new daemon, started in step 4, has loaded its lists and bound port 53. How long that takes depends on the size of the lists.

The migration moves the allow and deny rules you added in 0.41 into Custom Lists. The old allow and deny commands now refuse to write. Create a Custom List with warden custom-list and mount it on a profile with warden profile mount.

--upgrade needs no LAN range (--lan-cidr). It keeps your configuration where it is, and leaves your edits alone.

If you uninstalled Warden and kept its configuration, this page does not apply. --upgrade refuses that host, because no Warden binary or service unit is left to replace. Run the install without --upgrade: it adopts the kept configuration and checks it first. See Reinstall over a kept configuration.

The health check

The new daemon passes only if all three proofs hold, queried at 127.0.0.1:

• it answers on the listener;
• google.com resolves to a real IPv4 address;
• doubleclick.net is blocked with a 0.0.0.0 answer.

The check waits up to 300 seconds. If your setup answers blocked names in another way, or blocks everything, set the matching variable before you upgrade. See Environment variables. An upgrade that fails the check rolls back.

If your listen setting holds only a non-loopback address, the check cannot reach the daemon at 127.0.0.1. The new version fails the check after the wait. The rollback then asks the old daemon at the same address and gets no answer either, so the upgrade ends in RECOVERY REQUIRED with exit status 75. The old daemon runs, but the backup timer stays stopped: follow End states after exit 75. To avoid this, name the address when you upgrade:

curl -fsSL https://get.purge.cc | sudo WARDEN_UPGRADE_DNS_SERVER=192.0.2.10 sh -s -- --upgrade

The default listen, 0.0.0.0:53, is not affected.

When the check finds something to review

If the check finds a 0.41 rule that schema 5 reads differently, the upgrade stops before it changes anything and prints the command that lists the findings. Run these steps as root.

1. List the findings. The <candidate-elf> is the warden file in the unpacked bundle, /usr/local/lib/purge-warden/warden for a get.purge.cc install. <master> is your config file (/var/lib/purge-warden/config.toml on a 0.41 host):

   bash copy

   <candidate-elf> --config <master> migrate v4-to-v5 --check --json
      

   Each entry in findings has a code, a scope (where it applies), a cause and allowed_remedies.

2. Accept only the findings whose allowed_remedies include accept_v5_semantics. A finding that allows only defer cannot be accepted and keeps the upgrade refused.

3. Write a choices file, owned by root and not writable by group or others, for example /root/choices.json. Use the source_config_revision from the same output and one entry for each accepted finding:

   /root/choices.json

   json copy

   {"choices_version": 1, "planner_version": 1,
    "source_config_revision": "<source_config_revision from --check --json>",
    "finding_decisions": [{"finding_id": "<finding_id>", "decision": "accept_v5_semantics"}]}
      

4. Run the upgrade again with the file. Put the variable after sudo, not before it:

   bash copy

   curl -fsSL https://get.purge.cc | sudo WARDEN_UPGRADE_MIGRATION_CHOICES=/root/choices.json sh -s -- --upgrade
      

Do not change the configuration between the check and the upgrade. A change makes the file stale, and the check then fails with StaleChoices. List the findings again and write a new file.

Cluster secondaries on 0.41

A 0.41 cluster secondary cannot be upgraded in place. The upgrade refuses it with nothing changed, because its check reports cluster_local_policy_conflict. Take the node out of the cluster, upgrade both nodes, then join again:

1. On the secondary, still on 0.41, run warden cluster leave. If it asks for a resolver, run warden cluster leave --upstream <addr:port>.
2. Upgrade the secondary as a standalone node.
3. Upgrade the primary.
4. On the primary, run warden cluster create --migrate-legacy --san <primary address>, then warden cluster apply --preview-id <id>, then systemctl restart purge-warden. --san is the address the secondary uses to reach the primary. If the primary’s API is off or listens on loopback, add --api-listen <addr:port> with an address the secondary can reach.
5. On the primary, run warden cluster invite.
6. On the secondary, run warden cluster join --peer <primary https origin> --invitation-file <file>, then warden cluster apply --preview-id <id>, then systemctl restart purge-warden.

Upgrading a host whose daemon will not start

On a 0.42.0, 0.43.0 or 0.44.0 host (schema 5) the upgrade also accepts a daemon that systemd reports as failed. This recovers a host whose installed version no longer starts, for example one that stopped with receipt identity mismatch after a reboot. A 0.41.x host still needs a running daemon.

• activating means systemd is still retrying. Wait until systemctl is-active purge-warden.service prints active or failed, then upgrade.
• inactive means the daemon was stopped. Start it, then upgrade. If it fails to start, the upgrade accepts that.
• If the new version fails its check, the upgrade puts the old version back, starts it once and exits with the message the exact old ELF and unit were restored. An old version that cannot start fails again within about 30 seconds, as it did before.

Exit statuses

What a signal does

• Before the transaction commits, Ctrl-C, a hangup or a plain kill (SIGTERM) rolls the configuration back and starts the old version again.
• After the commit, the upgrade ignores these signals and completes. For a schema 5 host that is from the moment the health check passes. For a schema 4 host it is from the start of the migration finalize.
• A signal sent only to the install.sh process reaches install.sh and not the upgrade itself. The same happens with a hangup when install.sh itself leads the terminal session, or when sudo without use_pty leads it. The upgrade then runs to its end, and install.sh stops with 4 if the transaction completed.
• Only SIGKILL, a power loss or a crash can leave a host outside these statuses. See A killed or crashed upgrade.

End states after exit 75

Before it exits 75, the upgrade prints a message that tells you which case you are in. The full steps ship with the bundle, in docs/UPGRADE_RECOVERY.md. For a get.purge.cc install the file is at:

/usr/local/lib/purge-warden/docs/UPGRADE_RECOVERY.md

Start at “End states after exit 75” in that file. In short:

The manual warden migrate v4-to-v5 commands exist only as a recovery tool. They are never a replacement for install.sh --upgrade. They are also the route for a schema-4 configuration that uninstall.sh kept, because no installed Warden is left for --upgrade to work on. See Reinstall over a kept configuration.

A killed or crashed upgrade

SIGKILL, a power loss or a crash can stop the upgrade at any point. The old binary and unit can then remain only as files ending in .s6-old. While such a file exists, the next install.sh --upgrade refuses to start, before it changes anything, and prints the steps to recover. Follow “Killed or crashed upgrades” in UPGRADE_RECOVERY.md.

Recover before you run uninstall.sh. The uninstaller removes the .s6-old files, and after that the killed upgrade can no longer be recovered.

A command in a config directory may refuse with a message that names a .warden-migration fence. That fence belongs to an interrupted policy write or migration. Restart the daemon with systemctl restart purge-warden, which finishes or removes it. Never delete .warden-migration by hand.

Going back

• To 0.41.x. A failed upgrade, or one stopped by Ctrl-C, a hangup or SIGTERM before its last step, rolls the configuration back and starts 0.41.x again. After a crash or a power loss, follow “Killed or crashed upgrades” in UPGRADE_RECOVERY.md. Once the upgrade has finalized the migration, a 0.41.x binary cannot start on that configuration.
• To 0.43.0. After 0.44.0 has started once, 0.43.0 may not start on the same state directory. It can restart in a loop with enrollment policy snapshot changed during publication. A rollback by the upgrade puts the saved publication store back first. After a killed upgrade or an exit 75, the recovery file’s steps put the store back from the snapshot the upgrade printed as publication store snapshot: <dir>, before 0.43.0 starts. A completed upgrade deletes its snapshot, so after a completed upgrade there is no supported way back to 0.43.0 on the same state directory.
• To 0.44.0. Run the 0.44.0 installer in upgrade mode. From the unpacked 0.44.0 bundle, run sudo ./scripts/install.sh --upgrade --binary ./warden. Or run curl -fsSL https://get.purge.cc | sudo PURGE_VERSION=v0.44.0 sh -s -- --upgrade. The configuration stays schema 5, and archives written by 0.44.5 restore on 0.44.0. The 0.44.0 check refuses an entry in packs/ that is not a pack, such as an editor swap file: it exits with status 3 and changes nothing. Remove the entry first; warden status lists such entries as ignored. On the get.sh route that refusal comes after the bundle swap, so /usr/local/lib/purge-warden then holds the 0.44.0 bundle while 0.44.5 keeps running. Remove the entry and run the same command again. The bundle route leaves /usr/local/lib/purge-warden as it was.
• Before any downgrade, open the dashboard once (warden dashboard). If it shows Policy recovery, press r until the change is committed or aborted. And if you moved the configuration with warden config relocate, move it back first with --reverse (see below). Releases before 0.43.0 do not know the move.

Where your configuration lives

A fresh install of 0.43.0 or later keeps its configuration at /etc/purge-warden/config.toml. Files it includes, such as Custom List packs and list files imported from this machine, sit beside it. Downloaded list caches, data, logs, the node state and backup archives stay under /var/lib/purge-warden.

An upgrade does not move your configuration. A host installed before 0.43.0 keeps /var/lib/purge-warden/config.toml, and warden init refuses to create a second configuration under /etc while the old one exists. Everything works from the old place. Moving is an optional, offline step you run yourself with warden config relocate. On a host that still uses /var/lib/purge-warden/config.toml, the upgrade also prepares /var/lib/purge-warden/backups. If that path is a symlink, the installer warns and leaves it unchanged: it never follows the link, and the upgrade goes on.

Move the config to /etc

warden config relocate moves the configuration of an upgraded host from /var/lib/purge-warden to /etc/purge-warden. It is optional, and the upgrade never does it for you.

How it works:

• It runs offline and as root. Stop purge-warden and purge-warden-backup.timer first.
• Run the real binary, /usr/local/libexec/purge-warden/warden. The warden command on your PATH runs as the service user, which the move refuses.
• --config is refused. The two paths are fixed.
• Every file moves by rename, so it keeps its owner and mode. /etc and /var/lib/purge-warden must be on the same filesystem and mount. Otherwise the move refuses and changes nothing.
• Node identity and cluster pairing survive the move. List files imported from this machine move with the configuration. Downloaded list caches, state and backup archives stay in /var/lib/purge-warden.
• When restorecon is installed, the move relabels the new location.
• The move refuses a configuration that has no absolute [backup] dir. Set it first, for example dir = "/var/lib/purge-warden/backups".
• A run that stops part way keeps the node fenced and prints the one command that rolls it back. The daemon refuses to start until that command has run.

Steps

1. Take a backup. If you use Warden Nodes, also check that no node operation is in progress:

   bash copy

   sudo warden config backup
   warden node status
      

2. Only if [backup] dir is unset or relative, set an absolute one with sudo warden config edit, in the existing [backup] table if the file has one:

   config.toml

   toml copy

   [backup]
   dir = "/var/lib/purge-warden/backups"
      

   Then run sudo warden config lint.

3. Stop the daemon and the backup timer:

   bash copy

   sudo systemctl stop purge-warden-backup.timer purge-warden
      

4. Review the plan. A dry run runs every check and moves nothing:

   bash copy

   sudo /usr/local/libexec/purge-warden/warden config relocate --dry-run
      

5. Run the move:

   bash copy

   sudo /usr/local/libexec/purge-warden/warden config relocate
      

6. Check the result as the daemon user, before the service starts:

   bash copy

   test -f /etc/purge-warden/config.toml && test ! -e /var/lib/purge-warden/config.toml
   sudo -u purge-warden /usr/local/libexec/purge-warden/warden --config /etc/purge-warden/config.toml config lint
      

7. Start the service and the timer, and wait until the daemon answers:

   bash copy

   sudo systemctl start purge-warden purge-warden-backup.timer
   timeout 60 sh -c 'until warden status >/dev/null 2>&1; do sleep 1; done'
   warden status
      

   The start command returns before the daemon answers, so wait for warden status. The first normal start after the move confirms the new location.

After the move, warden config backup archives the set under /etc only. The archive directory stays where [backup] dir points.

Exit codes

warden config relocate exits 0 when it moved the set, rolled it back, or the dry run found nothing to refuse. It exits 1 when it refused, stopped, or the relabel failed. It exits 2 when the configuration does not load.

Refusals

A refusal prints refused (<name>): <reason>. Most refusals change nothing and end with Nothing was changed. Only a refusal during an incomplete run names the rollback command, because an incomplete run keeps the node fenced:

An incomplete run keeps this node fenced. Roll it back first: /usr/local/libexec/purge-warden/warden config relocate --reverse

Moving back

--reverse moves the configuration back to /var/lib/purge-warden. It also rolls back an incomplete forward run. Follow the same steps with --reverse, and run --reverse --dry-run first. The reverse dry run lists the /etc directories that the cleanup removes when they are empty.

sudo systemctl stop purge-warden-backup.timer purge-warden
sudo /usr/local/libexec/purge-warden/warden config relocate --reverse --dry-run
sudo /usr/local/libexec/purge-warden/warden config relocate --reverse
sudo -u purge-warden /usr/local/libexec/purge-warden/warden --config /var/lib/purge-warden/config.toml config lint
sudo systemctl start purge-warden purge-warden-backup.timer

Move back before you downgrade to a release that does not know the move.

See also

• Quick start: a first install.
• File paths: where Warden reads and writes.
• Environment variables: the WARDEN_UPGRADE_* settings.
• Backup & restore: take a backup before you upgrade.
• Troubleshooting: common problems after an install or upgrade.