AkurAI-Notes / .plan/epics/E07-deploy-versioning-changelog.md

Latest change f1d9b8cc6ff48113d891a2d95fd79b2c49f1f4ff - Initial plan and .plan directory by Claude Code

# E07 — deploy.sh release engine + versioning + changelog

## Goal

A `deploy.sh` that is a near-verbatim fork of AkurAI-Framework's own release
engine (NOT a stripped variant). Runs the full gate → bump → changelog → tag →
push → live publish cycle. Locks `VERSION`, `Cargo.toml [workspace.package]
version`, and `site/backend/page.json` version badge in lockstep.
Starts at `0.0.1`; `./deploy.sh patch` walks `0.0.2`, `0.0.3`, …

Same `deploy.sh` also installs:
- the Notes binary on `akurai-mail` as a systemd unit
- the local embedding server as a SEPARATE systemd unit (`akurai-notes-embed.service`)
- two nginx `location` blocks (`/` → :8099, `/mcp` → :8100)
- `/etc/akurai-notes/notes.env` (opens the OAuth `client_secret` from E03)
- TLS cert for `akurai-notes.olibuijr.com`

## Reference source

**Copy from:** `~/Projects/AkurAI-Framework/deploy.sh` (ver调剂 the change list
below).
**Do NOT copy:** `~/Projects/akurai-crm/deploy.sh` (CRM uses a stripped
release-with-no-publish variant — we want the full engine).

Read the full file once before starting:

```bash
cat ~/Projects/AkurAI-Framework/deploy.sh
```

## Steps

### 1. Fork the Framework deploy.sh into AkurAI-Notes/deploy.sh

Line-by-line changes:

| Framework deploy.sh | Notes deploy.sh |
|---------------------|------------------|
| `REPO="olibuijr/AkurAI-Framework"` | `REPO="olibuijr/AkurAI-Notes"` |
| `SITE_CHANGELOG="site/content/changelog.md"` | drop this var (Notes has no docs site) |
| `DOMAIN="akurai-framework.olibuijr.com"` (inside `ec2_publish`) | `DOMAIN="akurai-notes.olibuijr.com"` |
| `PORT=8094` | `PORT=8099` |
| `NAME="akurai-framework"` | `NAME="akurai-notes"` |
| `MUSL="target/x86_64_64-unknown-linux-musl/release/akurai"` | `MUSL="target/x86_64-unknown-linux-musl/release/akurai-notes"` |
| `cargo build --release --target … -p akurai` | `-p akurai-notes` |
| `[ -d site/frontend ]` exists-check | Same — Notes also has `site/frontend` |
| `akurai-ec2 deploy-binary "$NAME" "$MUSL" "$PORT" site` | Same shape (path relative to repo root) |
| `akurai-ec2 nginx-proxy "$DOMAIN" "$PORT"` | Replace with **custom nginx vhost write** (see §3) — Notes needs two locations, not one |
| The `cut_changelog` calls for both `CHANGELOG.md` and `SITE_CHANGELOG` | Drop the `SITE_CHANGELOG` call; only cut `CHANGELOG.md` |
| In the changelog helper, `unreleased` for repo = `"## [Unreleased]"` | Keep this verbatim (Notes uses the same Keep-a-Changelog dialect) |

Keep everything else exactly: the `say` / `die` shell helpers, the bump case
statement, the `gh repo create` block (creates `olibuijr/AkurAI-Notes` private
on the first run), the git push `-u origin HEAD` + push the `v$NEW` tag,
the `ec2_publish` invocation at the end.

### 2. Replace `akurai-ec2 nginx-proxy` with a custom vhost-write step

`akurai-ec2 nginx-proxy akurai-notes.olibuijr.com 8099` writes a single-
location vhost. Notes needs two locations — the default `/` and `/mcp`. Write
the vhost directly via ssh then reload nginx:

```bash
write_nginx_vhost() {
  ssh "$HOST" 'sudo bash -s' <<'NGINX'
set -euo pipefail
cat > /etc/nginx/sites-available/akurai-notes.olibuijr.com <<EOF
# Managed by AkurAI-Notes deploy.sh — do not edit by hand.
upstream akurai_notes_app  { server 127.0.0.1:8099; }
upstream akurai_notes_mcp { server 127.0.0.1:8100; }

server {
    listen 80;
    server_name akurai-notes.olibuijr.com;
    return 301 https://\$host\$request_uri;
}

server {
    listen 443 ssl http2;
    server_name akurai-notes.olibuijr.com;

    # SSL cert paths (managed by certbot — akurai-ec2 tls handles this)
    ssl_certificate     /etc/letsencrypt/live/akurai-notes.olibuijr.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/akurai-notes.olibuijr.com/privkey.pem;

    # App: everything NOT under /mcp.
    location / {
        proxy_pass http://akurai_notes_app;
        proxy_set_header Host \$host;
        proxy_set_header X-Real-IP \$remote_addr;
        proxy_set_header X-Forwarded-For \$proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto https;
    }

    # HTTPS MCP server: long-read-timeout JSON-RPC.
    location /mcp {
        proxy_pass http://akurai_notes_mcp;
        proxy_set_header Host \$host;
        proxy_set_header X-Real-IP \$remote_addr;
        proxy_set_header X-Forwarded-For \$proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto https;
        proxy_set_header Authorization \$http_authorization;
        proxy_read_timeout 600s;
        proxy_send_timeout 600s;
        proxy_buffering off;             # important for streaming-friendly responses
    }
}
EOF
ln -sf /etc/nginx/sites-available/akurai-notes.olibuijr.com /etc/nginx/sites-enabled/
sudo nginx -t && sudo systemctl reload nginx
echo "  [done] nginx vhost installed + reloaded"
NGINX
}
```

Note: the `\$` escapes the inner-bash `\$host` etc. correctly when passing
through SSH heredoc + remote bash. **Read carefully:** the local `<<EOF` lets
`$()` expand variables; for the nginx vars we want literal `${...}`, so we
escape with `\`. Use `<<'NGINX'` for the SSH heredoc (no local expansion inside
the outer remote script), `<<EOF` for the inner remote cat (so `$host`,
`$remote_addr` go to nginx unchanged, after the `\$` escape).

### 3. Add systemd unit install for two services

The Framework's deploy installs one unit; Notes installs two. Append to
`ec2_publish` after the binary deploy:

```bash
install_systemd_units() {
  ssh "$HOST" 'sudo bash -s' <<'UNITS'
set -euo pipefail

# 1) Main app
cat > /etc/systemd/system/akurai-notes.service <<EOF
[Unit]
Description=AkurAI-Notes (web app + REST API)
After=network-online.target akurai-notes-embed.service
Wants=network-online.target

[Service]
Type=simple
User=akurai
Group=akurai
ExecStart=/opt/akurai-notes/akurai-notes serve \
  --host 127.0.0.1 --port 8099 \
  --db /var/lib/akurai-notes/notes.db \
  --frontend-dir /opt/akurai-notes/site/frontend
EnvironmentFile=/etc/akurai-notes/notes.env
Restart=always
RestartSec=3
NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=full
ReadWritePaths=/var/lib/akurai-notes

[Install]
WantedBy=multi-user.target
EOF

# 2) MCP server (separate process so its auth + read-timeouts are decoupled)
cat > /etc/systemd/system/akurai-notes-mcp.service <<EOF
[Unit]
Description=AkurAI-Notes MCP server (POST /mcp JSON-RPC over HTTPS)
After=network-online.target akurai-notes.service
Wants=network-online.target

[Service]
Type=simple
User=akurai
Group=akurai
ExecStart=/opt/akurai-notes/akurai-notes mcp \
  --host 127.0.0.1 --port 8100 \
  --db /var/lib/akurai-notes/notes.db \
  --owner-sub \$AKURAI_NOTES_MCP_OWNER_SUB
EnvironmentFile=/etc/akurai-notes/notes.env
Restart=always
RestartSec=3
NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=full
ReadWritePaths=/var/lib/akurai-notes

[Install]
WantedBy=multi-user.target
EOF

# 3) Embedding server (CPU GGUF inference; OpenAI-compatible /v1/embeddings on :8081)
cat > /etc/systemd/system/akurai-notes-embed.service <<EOF
[Unit]
Description=AkurAI-Notes on-device embedding server (CPU GGUF, OpenAI-compat)
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
User=akurai
Group=akurai
# llama-server ships as /usr/local/bin/llama-server; the model file is at
# /opt/akurai-notes-embed/Qwen3-Embedding-0.6B-Q4_K_M.gguf.
ExecStart=/usr/local/bin/llama-server \
  --model /opt/akurai-notes-embed/Qwen3-Embedding-0.6B-Q4_K_M.gguf \
  --host 127.0.0.1 --port 8081 \
  --embedding
Restart=always
RestartSec=5

[Install]
WantedBy=multi-user.target
EOF

# User + dirs
if ! id -u akurai >/dev/null 2>&1; then
  useradd -r -m -d /opt/akurai -s /usr/sbin/nologin akurai
fi
mkdir -p /opt/akurai-notes /var/lib/akurai-notes /etc/akurai-notes
chown -R akurai:akurai /opt/akurai-notes /var/lib/akurai-notes /etc/akurai-notes

systemctl daemon-reload
systemctl enable --now akurai-notes-embed.service
systemctl restart  akurai-notes.service
systemctl restart  akurai-notes-mcp.service
echo "  [done] systemd units installed + restarted"
UNITS
}
```

If `llama-server` is NOT on the EC2 box yet, the `akurai-notes-embed.service`
install writes the unit but leaves it in `failed` state; print a clear
warning (not a deploy failure) and continue. Ólafur can install
`llama-server` separately — the Notes binary doesn't depend on it being up
(the substring-fallback path works without it).

Add the model fetch step:

```bash
ensure_embedding_model() {
  ssh "$HOST" 'bash -s' <<'MODEL'
set -euo pipefail
DIR=/opt/akurai-notes-embed
MODEL=Qwen3-Embedding-0.6B-Q4_K_M.gguf
URL=https://huggingface.co/Qwen/Qwen3-Embedding-0.6B-GGUF/resolve/main/$MODEL
[ -d "$DIR" ] || sudo mkdir -p "$DIR" && sudo chown -R akurai:akurai "$DIR"

if [ ! -f "$DIR/$MODEL" ] || [ "$(stat -c%s "$DIR/$MODEL")" -lt 200000000 ]; then
  echo "Fetching $MODEL from $URL …"
  # The EC2 box may not have curl/wget with HF cookie support — use bat-anything
  curl -fL "$URL" -o "/tmp/$MODEL" && sudo mv "/tmp/$MODEL" "$DIR/$MODEL"
fi
sudo chown akurai:akurai "$DIR/$MODEL"
MODEL
}
```

Verify the URL is correct by checking Hugging Face for an `Qwen3-Embedding-0.6B-GGUF`
repo before first run; if not present, fall back to `bge-small-en-v1.5-Q6_K.gguf`
or any small CPU embedding GGUF. The Notes binary doesn't care — it just
queries `/v1/embeddings`.

### 4. Provision `/etc/akurai-notes/notes.env` on first deploy

```bash
ensure_env_file() {
  ssh "$HOST" 'sudo bash -s' <<'ENV'
set -euo pipefail
F=/etc/akurai-notes/notes.env
if [ ! -f "$F" ] || ! grep -q '^AKURAI_NOTES_IDP_CLIENT_ID=' "$F"; then
  # The OAuth client_secret was stashed by E03 in /etc/akurai-notes/notes.env.pending
  PENDING=/etc/akurai-notes/notes.env.pending
  sudo touch "$F"; sudo chmod 0600 "$F"

  if [ -f "$PENDING" ]; then
    sudo cat "$PENDING" | sudo tee -a "$F" >/dev/null
  fi

  # Top up with non-secret config if missing
  grep -q '^AKURAI_NOTES_DB='            "$F" || echo "AKURAI_NOTES_DB=/var/lib/akurai-notes/notes.db"           | sudo tee -a "$F" >/dev/null
  grep -q '^AKURAI_NOTES_IDP_ISSUER='   "$F" || echo 'AKURAI_NOTES_IDP_ISSUER=https://auth.olibuijr.com'         | sudo tee -a "$F" >/dev/null
  grep -q '^AKURAI_NOTES_IDP_REDIRECT=' "$F" || echo 'AKURAI_NOTES_IDP_REDIRECT=https://akurai-notes.olibuijr.com/auth/callback' | sudo tee -a "$F" >/dev/null
  grep -q '^AKURAI_NOTES_ALLOW='        "$F" || echo 'AKURAI_NOTES_ALLOW=olibuijr@olibuijr.com'                  | sudo tee -a "$F" >/dev/null
  grep -q '^AKURAI_NOTES_EMBED_URL='    "$F" || echo 'AKURAI_NOTES_EMBED_URL=http://127.0.0.1:8081/v1/embeddings' | sudo tee -a "$F" >/dev/null
  grep -q '^AKURAI_NOTES_EMBED_MODEL='  "$F" || echo 'AKURAI_NOTES_EMBED_MODEL=qwen3-embedding-0.6b'              | sudo tee -a "$F" >/dev/null

  # MCP token: generate on first install; visible only to root/admin.
  if ! grep -q '^AKURAI_NOTES_MCP_TOKEN=' "$F"; then
    T="mcp_$(openssl rand -hex 32)"
    echo "AKURAI_NOTES_MCP_TOKEN=$T" | sudo tee -a "$F" >/dev/null
    echo "  [generated] AKURAI_NOTES_MCP_TOKEN (value in $F)"
  fi

  # MCP maps to Ólafur's user.
  grep -q '^AKURAI_NOTES_MCP_OWNER_SUB=' "$F" || echo 'AKURAI_NOTES_MCP_OWNER_SUB=olibuijr@olibuijr.com' | sudo tee -a "$F" >/dev/null
fi
ENV
}
```

### 5. Healthcheck at the end of `ec2_publish`

After the units restart, verify:

```bash
sleep 2
ssh "$HOST" 'curl -fsS http://127.0.0.1:8099/api/health' | jq .ok | grep -q true || die "/api/health failed"
ssh "$HOST" 'curl -fsS http://127.0.0.1:8100/mcp'        | grep -qE '405|method' || die "MCP /mcp didn't answer 405 on bare GET (expected)"

# Optional (embedding service up?):
ssh "$HOST" 'curl -fsS http://127.0.0.1:8081/v1/embeddings -H "Content-Type: application/json" -d "{\"input\":\"x\",\"model\":\"qwen3-embedding-0.6b\"}" | jq -re ".data[0].embedding | length"' \
    | head -1 || say "embed server not ready yet — semantic search will use substring fallback until it's up"

# Public URL:
curl -fsS https://akurai-notes.olibuijr.com/api/health | jq .ok | grep -q true \
    || say "Public URL not healthy yet — DNS/TLS may still be propagating"
```

TLS request failure (certbot run) does NOT fail the release — same as
Framework's `akurai-ec2 tls` graceful skip.

### 6. Full deploy.sh top structure

The full file composition (in order):

1. shebang + `set -euo pipefail`
2. `REPO`, `HOST`, `ROOT`, `cd`, `say` / `die` helpers
3. `ec2_publish()` function (subdivided):
   - `build_musl`
   - `ship_binary`
   - `ensure_env_file`
   - `ensure_embedding_model`
   - `install_systemd_units`
   - `write_nginx_vhost`
   - `run_tls` (via `akurai-ec2 tls`)
   - `healthcheck`
4. `cut_changelog()` (repo only — no site changelog)
5. `if [ "${1:-}" = "ec2" ]; then ec2_publish; exit 0; fi`
6. Bump argument parsing
7. Quality gate (fmt/clippy/test)
8. Version bump (lockstep)
9. CHANGELOG.md cut
10. `git add -A` + `git commit -q -m "release: v$NEW"` + `git tag -a "v$NEW" -m "v$NEW"`
11. `gh repo create` if no origin
12. `git push -u origin HEAD && git push origin "v$NEW"`
13. `ec2_publish`
14. `say "Released v$NEW ✓"`

### 7. Maintain `CHANGELOG.md` going forward

Initial `CHANGELOG.md` seeded in E01 with `## [Unreleased]`:
```
## [Unreleased]

### Added
- Initial pure-Rust workspace scaffolded from AkurAI-Framework's `akurai new`…
- On-device semantic search via Framework's `akurai-vector` + an OpenAI-compatible
  local embedding server
- AkurAI IDP SSO (authorization_code flow with `olibuijr@olibuijr.com` allowlist)
- HTTPS MCP server at /mcp with bearer-token auth
- Landing page + GPT-5.5 app icon set

### Changed
- Adopted lockstep versioning starting at `0.0.1` per Framework's `AGENTS_DEPLOY.md`.
```

Every work epic adds entries under `## [Unreleased]` as it lands. The epic
acceptance includes a "append this line to CHANGELOG.md Unreleased" step.

`./deploy.sh patch` then cuts `## [Unreleased]` into `## [0.0.1] - <date>`,
leaves a fresh `## [Unreleased]` on top, commits, tags, pushes, publishes.

### 8. Pre-release: make sure the initial commit has `VERSION=0.0.1` AND `Cargo.toml version = "0.0.1"`

E01 already did this. The first `./deploy.sh patch` bumps `0.0.1` → `0.0.2`.
For the v0.0.1 release (the bootstrap), Ólafur can either:

- Tag the bootstrap commit manually (`git tag -a v0.0.1 -m v0.0.1 && git push origin v0.0.1`) — then run `./deploy.sh ec2` to publish without bumping, OR
- Bump the version in `Cargo.toml` + `VERSION` to `0.0.1` first, commit it as
  `release: v0.0.0-bak` (sentinel), then immediately `./deploy.sh patch` which
  outputs `0.0.1`. The cleanest is the second approach; E01 already wrote the
  files with `0.0.1`. So the first deploy is `./deploy.sh patch` → `0.0.2`.

**For Ólafur to ship v0.0.1 specifically**, the easier path is: do not have a
prior tagged release; tag `v0.0.1` by hand first using
`git tag -a v0.0.1 -m v0.0.1 && git push origin v0.0.1` then
`./deploy.sh ec2` runs `ec2_publish` without bumping. The NEXT release is
`./deploy.sh patch` → `v0.0.2`. **Document this convention in `README.md`.**

## Files touched

```
deploy.sh                       ← NEW (fork of Framework deploy.sh)
CHANGELOG.md                    ← maintained (initial in E01)
README.md                       ← edited to document deploy.sh + v0.0.1 convention
```

## Acceptance

```bash
# Local script syntax + first stages:
bash -n ~/Projects/AkurAI-Notes/deploy.sh         # syntax check
! grep -q 'site/content/changelog' ~/Projects/AkurAI-Notes/deploy.sh && echo "PASS: no site changelog ref"

# Quality gate (assume all epics pass — we run the script in --dry-run-like mode later):
cd ~/Projects/AkurAI-Notes
cargo fmt --all -- --check
cargo clippy --all-targets --all-features -- -D warnings
cargo test --all

# Dry-check the script can identify the right musl target + binary name:
sed -n 's/.*MUSL="target.*release\/\(akurai[a-z-]*\)".*/\1/p' deploy.sh 2>/dev/null \
  | head -1
# expect: akurai-notes

# Confirm the repository's expected identity:
grep -q '^REPO="olibuijr/AkurAI-Notes"$' deploy.sh

# First live release (do this only when E01–E06 are verified):
# ./deploy.sh ec2   # publishes without bumping; or
# git tag -a v0.0.1 -m v0.0.1 && git push origin v0.0.1 && ./deploy.sh ec2
# Movement onward: "./deploy.sh patch" → v0.0.2
```

## Time

≈ 10 minutes if E01 already wrote the verbatim Framework deploy.sh shell.

## Depends on

E01 (workspace + VERSION lockstep setup), the nginx-write idiom existing already in the Framework's `akurai-ec2` (it does), and all epics E01–E06 passing their gate — otherwise the `cargo test --all` gate halts the release. **E07 is template-emittable at E01's bootstrap time** but only fires successfully once E01–E06 are done.