AkurAI-Build

public
Latest change c473d59271bebbbbbfd9250d2bc9ed003ba1d51a - ci: run verify on all branches, keep package/deploy main-only by Ólafur Búi Ólafsson
# PR queue recovery: binary rollback vs migration rollback

Scope: what actually happens today when a PR-branch verify run or a
production deploy needs to be walked back, and what is still only a plan.
Owned by the `feature/pr-ci-safety` slice (`.akurai.yml`, this doc, and
`tests/test_pr_pipeline.py`); it does not touch `src/`, `migrations/`, or
Cargo files.

## 1. Binary rollback — TESTED, exists in code today

Path: `deploy/service-deploy.sh` (in-sandbox CI job) hands a checksummed
candidate to the root-owned host agent `deploy/host/akurai-deploy-agent.sh`,
which:

1. Verifies the staged sha256 against the request file.
2. Copies the current `akurai` binary to `akurai.previous` before installing
   the candidate (`cp "$bin" "$bin.previous"`).
3. Restarts the service and polls `GET /api/health` up to 30 times
   (2s interval, 60s budget).
4. On health-gate failure, restores `akurai.previous` over the live binary
   and restarts again (`akurai-deploy-agent.sh` lines 67-69) — no human
   action required.

This is the same mechanism `src/ec2/deploy.rs::rollback` uses for the
`akurai-ec2 rollback` verb (backs up to `{binary}.prev`, requires the backup
to exist before allowing rollback). Both paths are exercised by the existing
`deploy-production` job (`branches: [main]`, gated on `package` which is
gated on `verify`) and require no schema assumptions — a binary swap is
inherently additive/replaceable and reversible as long as the previous
binary is retained, which the agent guarantees for exactly one generation
back.

Recovery is automatic and unattended: `deploy-production` only runs when
`verify` and `package` already passed, and the health gate is the pass/fail
signal, not a separate manual promotion step (this repo intentionally has no
approval gate on this job — see `.akurai.yml` header comment).

Limits observed in code, not assumed: only one prior binary generation is
retained (`.prev`/`.previous`), so two consecutive bad deploys before a
health check catches the first one will not be recoverable via this
mechanism alone.

## 2. Migration rollback — PLANNED, not yet implemented

`src/db.rs::migrate` applies `MIGRATIONS` (currently 001-010) forward inside
one transaction per boot, tracked by a ledger table and `PRAGMA
user_version`. There is no down-migration, no reverse-SQL table, and no CLI
verb that reverts a specific migration number. This is consistent with every
migration file inspected (001-010): they are additive (`ALTER TABLE ...ADD
COLUMN`, `CREATE TABLE`, `CREATE INDEX`) and none are destructive to
existing columns/rows.

Because of that additive-only pattern, "rollback" for a schema migration
today means: deploying a prior binary via the tested mechanism in section 1
while leaving the newer, additive schema in place. New nullable/defaulted
columns and new tables introduced by a migration are inert to an older
binary that doesn't reference them — this is a design consequence of the
additive convention, not a verified runtime property, since no automated
test exercises "old binary against new schema" today.

If a future migration needs to be destructive (drops a column, rewrites
data) the additive assumption above breaks and a tested down-path would be
required before that migration ships; this doc does not claim such a path
exists.

Coordination note: this slice does not add migration 011 or PR persistence —
that is owned by the primary application-integration builder (t_2578e75e /
t_2667e5e8 parent). If migration 011 stays additive, section 2's reasoning
continues to hold without further doc changes; if it is not additive, the
primary builder should update this section rather than assume it.

## 3. What is NOT tested

- Reverting a specific already-applied migration (no down-SQL exists).
- Recovering from two consecutive bad production deploys (only one `.prev`
  binary generation is kept).
- Any cross-check that an older binary genuinely tolerates a newer additive
  schema at runtime (assumed from the migration authors' additive
  discipline, not verified by an integration test).