AkurAI-Build / README.md

Latest change e8aa34a9be50d7d7ea72f66991ff82dfca7d23e1 - OLI-276: Git over SSH, forks and mirrors, code search through akurai-index, repository insights by Ólafur Búi Ólafsson

# AkurAI Build

A small Git-native CI/CD system built with pinned Rust 1.97: one Rust binary for repositories, pipelines, Docker/native jobs, configurable Titan build workers, build matrices, logs, artifacts, protected deployments, webhooks, filterable CLI/API queries, and a responsive MiniJinja workbench.

It keeps the useful center of Jenkins, GitLab CI, GoCD, Concourse, and Woodpecker with a minimal authenticated Smart HTTP Git host, but without plugin markets, cluster control planes, or frontend frameworks.

## Foundation

AkurAI Build starts from [Bunfork](https://github.com/olibuijr/bunfork)'s proven shape:

- Rust single binary
- encrypted SQLCipher state and ordered migrations
- bounded Axum/Hyper HTTP serving
- MiniJinja templates and dependency-free Ponytail UI
- strict CSP, constant-time tokens, protected files, and hardened systemd
- digest-recorded artifacts and rollback-oriented deployment

Bunfork remains a supported artifact target; AkurAI Build replaces its unrelated vector/demo application surface with CI/CD.

## Production topology

```text
https://akurai-build.olibuijr.com
        │ TLS
        ▼
EC2 Nginx / AkurAI-VPN
        │ private 100.88.0.0/16
        ▼
Titan 100.88.0.9:3210
  ├── akurai-build-service (HTTP, AKURAI_WORKERS=0)
  ├── akurai-build-worker (execution, AKURAI_WORKERS=1)
  ├── Docker / BuildKit
  ├── encrypted SQLite
  └── workspaces, cache, artifacts
```

All releases use the MCP-controlled immutable pipeline:

```text
local commit → repo_sync → run_queue(exact SHA) → promote production → persisted success
```

The protected self-pipeline consumes the package artifact, replaces only the
service container, verifies service and worker health, then atomically installs
the packaged stdio MCP binary and shared CI helpers. Any failed step rolls the
service back. Worker-image replacement remains a separate host-maintenance
operation so a running worker never replaces itself.

## Pipeline

Commit `.akurai.yml`:

```yaml
version: 1
jobs:
  - name: test
    image: oven/bun:1.3
    network: true
    run:
      - bun install --frozen-lockfile
      - bun test
    matrix:
      platform: [linux/amd64, linux/arm64]
    cache: [.bun]

  - name: package
    needs: [test]
    image: oven/bun:1.3
    run: bun build src/index.ts --compile --outfile dist/app
    artifacts: [dist/**]

  - name: deploy
    needs: [package]
    executor: native
    environment: production
    approval: true
    branches: [main]
    secrets: [DEPLOY_TOKEN]
    run: akurai-ec2 release --mode publish --yes
```

Jobs default to depending on the previous job; set `needs: []` for an independent root. Caches are isolated by registered repository and shared by its jobs. Matrix axes expand to at most 32 variants and become `AKURAI_MATRIX_<AXIS>` variables. `platform` also selects Docker's `--platform`.

Docker jobs run with dropped capabilities, no-new-privileges, fixed CPU/memory/PID limits, a read-only root, writable workspace/tmp, host UID/GID, and no network unless `network: true`. Linux `amd64`/`arm64` variants use Titan Docker with pinned `tonistiigi/binfmt:qemu-v10.2.3-68` QEMU registration, rechecked by every deploy. Native jobs are disabled unless the repository is trusted and `AKURAI_ALLOW_NATIVE=1` is set.

Declared secrets resolve only from `AKURAI_SECRET_<NAME>`, are omitted from Docker arguments, and are exact-value-redacted from retained logs. Repository writers with secret-bearing jobs are privileged because build code can intentionally transform or exfiltrate a secret.

## Operator CLI and agent MCP

Every CLI command emits one JSON envelope for operator use. Agents use the configured AkurAI Build MCP tools for repository, run, artifact, approval, and deployment state; they never substitute CLI or raw stdio calls.

```sh
akurai-build doctor
akurai-build repo add app https://github.com/org/app.git --branch main
akurai-build repo host app /home/olafurbui/Projects/app --branch main
akurai-build repo sync app /home/olafurbui/Projects/app
akurai-build repo list --search app --visibility private
akurai-build run app --git-ref main
akurai-build runs --repo app,worker --status failed,interrupted --trigger manual,webhook --search compiler --limit 20
akurai-build show 42
akurai-build wait 42 --timeout 3600
akurai-build logs 42 --failed
akurai-build retry 42
akurai-build promote 42 production
akurai-build artifact get 7 ./artifact.bin
akurai-build workers
```

The CLI remains an operator and local-development surface. `repo host` creates a bare mirror below the protected AkurAI Build data root and atomically switches the CI registration to it; `repo sync` refreshes that mirror from its trusted Titan checkout. `runs` accepts composable repository, status, trigger, ref, and text filters. Agent publication and deployment use the matching configured MCP tools and require a persisted `succeeded` state.

### MCP control plane

The configured `akurai-build` stdio server exposes the full MCP-controlled workflow:

- repository registration, hosting, synchronization, visibility, branch heads, directory trees, and bounded file blobs;
- immutable run queueing, querying, waiting, retrying, canceling, logs, and protected-environment promotion;
- artifact retrieval, worker status, diagnostics, pipeline initialization, and semantic releases.

`akurai_repo_branches` returns the exact SHA for every branch head. `akurai_repo_tree` accepts a branch or immutable commit and a directory path; `akurai_repo_blob` reads a bounded UTF-8 file at a ref and path. The maintained launch entrypoint is `deploy.sh mcp`; clients use the configured `mcp__akurai_build_*` tools rather than invoking it directly.

### Hosted Git

Hosted repositories are available at `https://akurai-build.olibuijr.com/git/<name>.git`. Smart HTTP accepts the controller bearer token, a signed-in browser session, a personal access token or a deploy key (`Authorization: Bearer abt_…`, or git's `Basic <anything>:abt_…`); reads need the `read` role on the repository and `git-receive-pack` needs `write`. Tokens must be supplied through a protected credential helper or maintained command, never a URL, command argument, source file, or log. CI uses the local bare mirror directly, so builds do not depend on public DNS or the edge proxy.

### Git over SSH, forks, mirrors, code search and insights (OLI-276)

**SSH.** Each user registers OpenSSH public keys (ed25519, ecdsa, rsa ≥ 2048) on
`/app/settings/ssh-keys` or through the API; the forge rewrites the file named by
`AKURAI_BUILD_AUTHORIZED_KEYS` after every change, one `restrict`ed line per key whose
forced command is `akurai git-shell --key-id N`. That shell accepts only
`git-upload-pack` and `git-receive-pack` on a hosted repository, applies the Smart HTTP
roles (`read` to fetch, `write` to push, public repositories readable by any key holder,
`AKURAI_BUILD_ADMIN_EMAILS` admin everywhere), hands `REMOTE_USER` and the forge
callback to the hooks (branch protection and webhooks work as over HTTPS) and records
`last_used_at`. sshd needs the file in `AuthorizedKeysFile`; the clone address shown on
the repository page comes from `AKURAI_BUILD_SSH_ADDRESS` (`user@host`).

```text
GET    /api/ssh-keys                 the caller's keys (session or PAT)
POST   /api/ssh-keys                 {name, public_key}
DELETE /api/ssh-keys/{id}
```

**Forks and mirrors.** A fork is a hosted bare copy that remembers `fork_of` and can
fast-forward its default branch from the upstream (never past its own commits); the
forker becomes `admin` of the fork. A mirror is a hosted copy of an external URL that
pulls every branch and tag, on demand and every ten minutes (created by forge admins).

```text
POST   /api/repos/{repository}/fork      {name}            (read on the source)
GET    /api/repos/{repository}/forks                       (the forks the caller may read)
POST   /api/repos/{repository}/sync                        (write; fork or mirror)
POST   /api/mirrors                      {name, url, branch?}   (controller token or forge admin)
```

**Code search.** `GET /api/search?q=&repo=&k=` and `/app/search` ask akurai-index
(`AKURAI_BUILD_INDEX_URL`, token in `AKURAI_BUILD_INDEX_TOKEN_FILE`) and keep only hits
from repositories the caller may read; each hit links to the forge's file view.

**Insights.** `GET /api/repos/{repository}/insights?days=90` and the Insights tab
(`/app/repositories/{repository}/insights`, public repositories also at
`/repos/{repository}/insights`) aggregate the default branch: commits and lines per
contributor, commits per week and the most changed files.

### Users, roles, tokens and deploy keys

Identities are AkurAI Login users (their email is the subject); the controller token and
signed-in forge admins hold `admin` everywhere. Per repository:

```text
GET    /api/repos/{repository}/collaborators                 {subject, role: read|write|admin}
PUT    /api/repos/{repository}/collaborators                 (admin on the repository)
DELETE /api/repos/{repository}/collaborators/{subject}
POST   /api/tokens        {name, role, repository?, ttl_secs?, subject?}   a PAT for the signed-in user
                           (or for `subject` with the controller token); the plaintext is returned once
GET    /api/tokens        the caller's tokens (controller token: ?subject=)
DELETE /api/tokens/{id}
POST   /api/repos/{repository}/deploy-keys   {name, role: read|write, ttl_secs?}   (admin on the repository)
GET    /api/repos/{repository}/deploy-keys
DELETE /api/repos/{repository}/deploy-keys/{id}
```

A PAT is capped by its holder's collaborator role (and by its repository when scoped);
a deploy key holds its own role on one repository only. Tokens expire after at most a
year, can be revoked, and only their SHA-256 is stored. `REMOTE_USER` is handed to git
hooks, so the audit trail names the pusher.

### Branch protection

Every hosted repository gets a `pre-receive` hook (installed when it is hosted and at every
start) that asks the forge about each ref update: `POST /api/repos/{repository}/protection/check`
with `{reference, old, new, fast_forward}`. Rules live per branch:

```text
GET    /api/repos/{repository}/protection            rules
PUT    /api/repos/{repository}/protection            {branch, allow_force_push, allow_delete, require_green, require_review}
DELETE /api/repos/{repository}/protection/{branch}
```

The default branch of every repository starts with the default rule: no force-push, no
deletion; `require_green` (a succeeded run on the pushed commit) and `require_review` (the
commit is the merge of a pull request with an approval and no outstanding request for
changes) are opt-in. The pull-request merge endpoint applies the base branch's rule to the
pull request's head commit and reviews. Pushes by the forge user straight into the bare
repository bypass the hook; pushes over Smart HTTP cannot.

### Tags and releases

`GET /api/repos/{repository}/tags` lists the repository's tags (newest first, annotated
tags peeled to their commit). A release is a tag with a name, notes and attached assets:

```text
GET    /api/repos/{repository}/releases
POST   /api/repos/{repository}/releases                  {tag, name?, notes?}   (write; the tag must exist)
GET    /api/repos/{repository}/releases/{tag}
DELETE /api/repos/{repository}/releases/{tag}            (admin)
POST   /api/repos/{repository}/releases/{tag}/assets     {artifact_id, name?}   copies a run artifact in
GET    /api/repos/{repository}/releases/{tag}/assets/{name}   download (public repositories need no token)
```

Pages: `/repos/{repository}/releases` (public repositories) and
`/app/repositories/{repository}/releases` list releases with their notes and assets and the
tags with links to their commits. Assets live under `<data root>/release-assets/`.

### Notifications

Every signed-in person (and every PAT holder) has a feed: review requests
(`reviewers: [...]` when opening a pull request), submitted reviews (to the author),
merged pull requests (to author and reviewers) and failed runs (to the repository's
writers and admins plus the forge admins). `GET /api/notifications[?unread=true]`,
`POST /api/notifications/{id}/read`, `POST /api/notifications/read-all`, and the page
`/app/notifications`. With `AKURAI_BUILD_MAIL_FROM` and `AKURAI_BUILD_MAIL_CMD` (a command
that reads the message on stdin) each notification is also mailed once.

### Issue linking (Paperclip)

The forge keeps no issues, labels or milestones: Paperclip is the tracker, and its
projects, epics and labels are the taxonomy (decision, OLI-274). Commits and pull
requests that mention a Paperclip issue id (`OLI-123`; `fixes OLI-123` marks it as
fixing) get a comment on that issue with a link back to the commit or pull request,
once per commit and issue. Set `AKURAI_BUILD_PAPERCLIP_URL` and
`AKURAI_BUILD_PAPERCLIP_TOKEN_FILE` (a board API key) in the forge's environment;
without them links are still recorded locally and nothing is posted.

### Outbound webhooks

Per repository, admins subscribe URLs to `push`, `pull_request` (opened, closed,
reopened, reviewed, merged) and `run` (finished) events:

```text
GET    /api/repos/{repository}/webhooks
POST   /api/repos/{repository}/webhooks            {url (https), secret, events: [...]}
PATCH  /api/repos/{repository}/webhooks/{id}       {active}
DELETE /api/repos/{repository}/webhooks/{id}
GET    /api/repos/{repository}/webhooks/{id}/deliveries?limit=
POST   /api/repos/{repository}/webhooks/{id}/deliveries/{delivery}/redeliver
```

Deliveries are JSON POSTs `{event, repository, at, data}` with `X-AkurAI-Event`,
`X-AkurAI-Delivery` and `X-AkurAI-Signature-256: sha256=<HMAC-SHA256 of the body with the
secret>`. A delivery loop sends due deliveries every few seconds; failures retry after
30 s, 2 min, 10 min and 1 h, then the delivery is marked failed and can be redelivered.
Pushes are reported by the `post-receive` hook installed next to `pre-receive`.

## Git webhooks

Use:

```text
POST https://akurai-builds.olibuijr.com/api/hooks/<repository-name>
```

Supported authentication:

- GitHub `X-Hub-Signature-256`
- Gitea `X-Gitea-Signature`
- GitLab `X-Gitlab-Token`

The secret is the protected `webhook.token`; do not expose it in commands or documentation. Push payloads supply `ref` and the immutable commit. Manual/UI/API mutations require the separate bearer API token.

## State and recovery

- A run stores its resolved immutable commit before reading `.akurai.yml`.
- Jobs execute fail-fast in dependency order; all variants of a dependency must succeed.
- Artifacts are bounded, SHA-256 recorded, copied without symlinks/traversal, and served only after authorization.
- A protected deployment records the exact upstream artifact IDs and digests before execution.
- Every Titan worker persists its status, capacity, heartbeat, current run, and completed-run count; restart recovery marks stale workers offline before registering the configured pool.
- Compose restarts failed service/worker containers. The protected `.akurai.yml` deployment replaces only the service, verifies both health endpoints, installs the packaged MCP/host helpers, and rolls the service back on failure.

Run evidence is the system's operational knowledge. It supports debugging and safe iteration without allowing unreviewed self-modifying code.

## Limits kept intentionally

The first release uses one Titan scheduler with a configurable local worker pool and SQLite. Each worker owns one run at a time; jobs inside a run remain dependency-ordered and isolated. Linux container matrices are complete; native Windows/macOS and remote Windows-container builds wait for a signed artifact-streaming agent protocol rather than pretending remote bind mounts are portable. Object storage, provider commit-status adapters, and role-based multi-user accounts are deferred—not stubbed.

## Development and release

Run `cargo fmt --all -- --check`, Clippy with warnings denied, and `cargo test --workspace` locally. Commit only intended paths, publish with `akurai_repo_sync`, require the hosted branch SHA to match, then queue that exact commit. Production changes run only through the repository's approval-gated `.akurai.yml`; wait, promote explicitly, require persisted success, and verify the changed behavior.