Headscale and Headplane · Architecture
One Headscale Deployment and one Headplane Deployment share a hostname but use separate local SQLite databases. Each database lives on disposable local storage with Litestream replication to the shared NFS backup PVC. Startup restores the last backup before serving. Headplane calls the internal Headscale API using a mounted API key; browser login uses OIDC.
Component boundaries
- Identity / clients
- Tailscale client → OIDC
- Headplane browser → OIDC
- Single processes
- 1 Headscale / SQLite
- 1 Headplane / SQLite → Headscale API
- Recovery
- Litestream copies both databases
- NFS backups → startup restore
State and recovery contract
Preserve both Litestream backup directories, Headscale configuration and key material included in the storage arrangement, the Headplane cookie secret, and sealed OIDC/API credentials. A surviving backup is needed when the disposable SQLite volume is recreated.
Availability and failure behavior
Availability classification: Not continuously HA: Headscale and Headplane each recover by restarting one process.
This is an interpretation of the checked-in configuration, assuming the declared replicas are healthy, separated as intended, and their required dependencies are reachable. It is not a live-health result or a completed failure drill.
What supplies redundancy
| Component | Declared layout | Mechanism |
|---|---|---|
| Headscale | 1 Recreate Deployment pod | No second server is ready to take over control-plane requests. |
| Headplane | 1 Deployment pod; Recreate | The management interface has its own single-process outage. |
| Databases | Two separate local SQLite databases | Litestream copies each database into its NFS backup directory. |
| Backup target | One external NFS service | Replacement pods restore the latest available copy before serving. |
How a failure is handled
Kubernetes must detect the failure and start a replacement on an eligible node. Initialization restores the relevant SQLite database from NFS, then the process starts with its original configuration and credentials. That is restart-and-restore recovery, not routing to a hot standby.
Failure scenarios
| Failure | Expected behavior and remaining dependency |
|---|---|
| One main worker | If it hosts Headscale, registration and control updates stop until recovery. If it hosts Headplane, browser administration stops independently. Already-established client tunnels may continue using existing peer state, but new enrollment, policy distribution, or reconnection must not be assumed available. |
| RTX worker only | No dedicated voter is involved. Headscale startup also depends on OIDC discovery under the checked-in only-start-if-available setting, so Authentik must be reachable during recovery. |
| A second failure before recovery | Outside the stated single-failure envelope; assess remaining data copies, quorum, endpoints, and capacity before further maintenance. |
An RTX worker VM failure is not the same as an RTX Proxmox host failure. The latter also removes its control-plane VM and the configured API address. Read the physical failure-domain explanation before making a whole-host HA claim.
Upgrades and voluntary maintenance
Single-replica updates interrupt the corresponding service; Headplane uses Recreate. Do not scale SQLite writers to two as an HA shortcut without a supported coordination/storage redesign.
What prevents a stronger HA claim
Unreplicated recent SQLite changes may be absent from the restored backup. NFS loss prevents dependable backup/restore. Restoring both applications requires both databases, not just the Headscale one.
What would improve the availability contract
State a measured recovery-time and recovery-point objective, protect NFS and sealing material, and rehearse both restores. Continuous service availability would require a supported application/storage topology beyond the current singleton design.
These are operational/design requirements, not changes made to the deployment by this documentation. The shared availability reference explains election, replication, durability, recovery time, and shared dependencies; the manifest links below identify this service’s source.
Configuration ownership
apps/headscale/ owns this service’s Kubernetes resources. Hostnames, addresses, versions, node names, and volume sizes belong in apps/variables.yaml. Runtime settings stored in a product database need that product’s backup and administration procedure.
app.yamlcertificate.yamlconfig.yamlheadplane-config.yamlheadplane-secret.yamlheadplane.yamlheadscale.yamloidc-secret.yamlstorage.yaml
Continue
Read the deployment guide for dependency order, initial credentials, and integration work. The official documentation explains the upstream product; the topology above describes this repository.
Headscale uses a single Recreate Deployment restricted to the two general workers. Headscale and Headplane tolerate NotReady/unreachable nodes for 60 seconds before eviction. This permits replacement after a powered-off worker fails without waiting for a StatefulSet ordinal to clear. It does not fence a partitioned writer: confirm the old worker is powered off before takeover when connectivity is uncertain. RTX is excluded.
Before the controller migration, a Sync hook restores the existing Litestream backup into scratch storage using a read-only backup mount. An existing identity requires a successful restore; a missing identity alongside a database backup blocks migration. The Noise identity key and credentials remain unchanged. This provides recovery with an outage, not continuous availability.