AkurAI-Framework / AGENTS.md

Latest change 5458532ad0ba451030d0554d03be8ff383ccddb8 - feat(ws,http): pure-std WebSocket (RFC 6455) + Server-Sent Events by Ólafur Búi Ólafsson

# AGENTS.md — AkurAI-Framework

> The map. Read this first. Every subsystem is a small, single-responsibility
> crate; every crate is many small files. Land in the right file via the tables
> below — don't grep blind.

## What this is

A **single-binary, pure-Rust, zero-runtime-dependency** full-stack web framework
in the spirit of PocketBase: embedded relational database, auto-generated REST
API from a declarative schema, built-in semantic search, and a from-scratch
client-first frontend toolchain. Copy one binary to a box, run it, done.

## Constitutional principles (do not violate)

1. **Zero runtime dependencies.** The shipped binary links NO external crates —
   `std` only. Internal workspace crates are fine. Dev/test/bench tooling
   (proptest, criterion, Playwright) is allowed; it never enters the binary.
2. **TLS terminates at the edge** (Caddy/nginx). The server speaks plain HTTP on
   localhost. std cannot do TLS and we will not pull a crate for it.
3. **Client-first.** The browser does rendering, routing, and state. The server
   serves the data API and does SSR only for SEO / dynamic content that needs it.
   Keep the server lean.
4. **No-build frontend.** Native ES modules + import maps + web components +
   server-rendered HTML. NO JS bundler/transpiler. Win DX/memory by subtraction.
5. **Declarative source of truth.** `backend/collections.toml` defines the data
   model; the API, validation, migrations, and embeddings all derive from it.
6. **Self-describing.** `GET /api/_meta` emits a machine-readable manifest of
   every generated route. Agents discover the whole surface in one call.
7. **TDD, always.** Failing test first, every phase. No subsystem advances
   without its test layer (unit → property → integration → e2e/visual).
8. **Performance is a gate, not a hope.** Budgets (cold start, idle memory,
   hot-reload latency, throughput) are asserted; regressions fail the release.
9. **Small scoped files.** One responsibility per file. If a file grows two
   concerns, split it and index it here.
10. **Docs are first-class.** Maintain a `/docs` documentation site in a
    Next.js/Vue-style layout (sidebar nav, sectioned guides + API reference) and
    keep it updated as the framework evolves — every new crate or feature ships
    with its docs in the same commit.
11. **Data updates without a rebuild — essential.** All application data lives
    in the runtime data store (crash-safe, WAL-style durability) and editable
    context files (e.g. `backend/page.json`), never compiled into the binary.
    Changing data must NEVER require recompiling or redeploying the binary.

## Crate map

| Crate | Owns | Status |
|-------|------|--------|
| `crates/http` | HTTP/1.1 server, request/response, thread pool, connection upgrade, SSE — pure std | 🟢 working |
| `crates/ws` | WebSocket server (RFC 6455): handshake, framing, sha1+base64, conn — pure std | 🟢 working |
| `crates/cli` | `akurai new \| serve \| dev` — bootstrap + lean static server | 🟢 working |
| `crates/json` | std-only JSON parse/serialize/value (int/float preserved) | 🟢 working |
| `crates/storage` | single-file pager ✅, node codec ✅, CoW B+tree + atomic meta + free list ✅; blobs/rebalance next | 🟢 working |
| `crates/sql` | lexer → parser → planner → executor, catalog, types, txn/MVCC | ⬜ planned |
| `crates/vector` | embedding client, vector store, cosine + HNSW semantic search | ⬜ planned |
| `crates/api` | schema, auto-router, filter/rules DSL, auth, records, `/_meta` | ⬜ planned |
| `crates/template` | std-only SSR template engine (vars, if/for/include) | 🟢 working |
| `crates/markdown` | std-only Markdown → HTML (docs + changelog) | 🟢 working |
| `crates/css` | from-scratch atomic/utility CSS engine (Tailwind-class), dev+build | ⬜ planned |
| `crates/server` | (folded into `crates/cli` for now) SSR-on-demand later | ⬜ planned |
| `bench` | criterion micro-benchmarks + whole-stack load gen + perf budgets | ⬜ planned |

### `crates/http` files

| File | Responsibility | Status |
|------|----------------|--------|
| `src/request.rs` | parse the HTTP request head (method, target, version, headers) | ✅ |
| `src/lib.rs` | crate root, module exports | ✅ |
| `src/response.rs` | build/write HTTP responses (`to_bytes` + head-only `to_head_bytes`) | ✅ |
| `src/reply.rs` | `Reply` (Response \| Upgrade) — the connection-hijack seam | ✅ |
| `src/server.rs` | `TcpListener` accept loop; upgrades go to a dedicated thread | ✅ |
| `src/pool.rs` | std thread pool (`mpsc` + workers) | ✅ |
| `src/sse.rs` | Server-Sent Events stream (`text/event-stream`, spec framing) | ✅ |

### `crates/ws` files

| File | Responsibility | Status |
|------|----------------|--------|
| `src/lib.rs` | crate root + `upgrade(req, on_conn)` entry (101 or 400) | ✅ |
| `src/handshake.rs` | recognise WS upgrade + compute `Sec-WebSocket-Accept` | ✅ |
| `src/sha1.rs` | from-scratch SHA-1 (handshake only) + FIPS test vectors | ✅ |
| `src/base64.rs` | from-scratch base64 encode (RFC 4648) | ✅ |
| `src/frame.rs` | RFC 6455 §5 frame read/write, masking, length, limits | ✅ |
| `src/message.rs` | `Message` enum + RFC 6455 §7.4.1 close codes | ✅ |
| `src/conn.rs` | `WsConn`: reassembly, auto ping/pong, close handshake | ✅ |
| `tests/echo.rs` | end-to-end: real handshake + masked echo + close | ✅ |

### `crates/json` files

| File | Responsibility | Status |
|------|----------------|--------|
| `src/value.rs` | `Value` enum (order-preserving objects) + serializer | ✅ |
| `src/parse.rs` | recursive-descent parser (escapes, unicode, int/float) | ✅ |
| `src/error.rs` | parse error with character position | ✅ |

### `crates/storage` files

| File | Responsibility | Status |
|------|----------------|--------|
| `src/pager.rs` | single-file fixed-size page read/write/allocate + `sync` | ✅ |
| `src/node.rs` | B+tree node byte layout: leaf + interior encode/decode | ✅ |
| `src/meta.rs` | double-buffered checksummed meta page + inline free list | ✅ |
| `src/tree.rs` | CoW `BTree`: get/insert/delete/range, splits, atomic commit | ✅ |
| `tests/pager.rs` | integration: round-trip, persistence, durability, bounds | ✅ |
| `tests/tree.rs` | integration: torn-commit recovery, bulk reopen, range | ✅ |
| _next_ | underfull-node merge/rebalance, blobs (overflow pages), MVCC | ⬜ |

### `crates/cli` files

| File | Responsibility | Status |
|------|----------------|--------|
| `src/main.rs` | arg parsing + command dispatch (`new`/`serve`/`dev`/`version`) | ✅ |
| `src/cmd_new.rs` | bootstrap a project from embedded templates | ✅ |
| `src/cmd_serve.rs` | static server + `/api/health`, `/api/_meta`, SPA fallback | ✅ |
| `src/mime.rs` | file-extension → MIME mapping | ✅ |
| `assets/*` | embedded starter page + config/collection templates | ✅ |

## Directory layout

```
AkurAI-Framework/
├── AGENTS.md            ← this file (the index)
├── CHANGELOG.md         ← Keep-a-Changelog; deploy.sh cuts releases
├── VERSION              ← starts 0.0.1; deploy.sh bumps lockstep
├── deploy.sh            ← gate → bump → changelog → tag → push (creates repo)
├── Cargo.toml           ← workspace root
├── crates/              ← one subsystem per crate, small files
├── frontend/            ← native-ESM SPA + admin UI (embedded at build)
├── backend/             ← collections.toml, rules/, migrations/ (declarative SSOT)
├── data/                ← store.db + blobs/ (gitignored)
├── site/                ← the framework's OWN website (the framework serves it):
│                          frontend/ (landing, docs.html, components.html) +
│                          content/ (docs/*.md, changelog.md) + backend/page.json.
│                          Deployed to akurai-framework.olibuijr.com via deploy.sh ec2.
└── tests/e2e/           ← Playwright visual scripts + screenshot snapshots
```

## Semantic search (built-in, all collections)

Embedding model runs externally (OpenAI-compatible endpoint, e.g. Titan
EmbeddingGemma at `:8081`). The framework owns storage + search in pure Rust:
auto-embed configured text fields on write → store vector → rank by similarity.
Brute-force cosine first (fine at small/medium scale), HNSW later for scale.
Every collection gets `?search=<text>` for free.

## Testing & release

- **Unit:** inline `#[cfg(test)]` per module.
- **Property/fuzz:** proptest against SQL parser + B-tree invariants (dev-dep).
- **Integration:** `tests/` per crate (real DB files, real HTTP).
- **E2E/visual:** Playwright in `tests/e2e/` with screenshot snapshots; locally
  also verify with Interceptor (real Chrome).
- **Perf:** `bench/` with criterion + a load generator; budgets enforced in `deploy.sh`.
- **Release:** `./deploy.sh [patch|minor|major]` runs the gate, bumps VERSION +
  crate manifests, cuts the CHANGELOG, tags `vX.Y.Z`, pushes (creating the GitHub
  repo on first run).

## Roadmap (build order — each gated by its tests)

1. HTTP core + JSON
2. Storage engine (CoW B+tree, crash-safe) — the bedrock
3. SQL-A: single-table SELECT/INSERT/UPDATE/DELETE, WHERE, ORDER BY, LIMIT
4. SQL-B: indexes + index selection, JOIN, GROUP BY/aggregates
5. SQL-C: optimizer, hash joins, subqueries, txn/MVCC hardening
6. Vector/semantic search
7. Auto-API layer (schema → routes → `/_meta`)
8. Auth + relations + SSE realtime + uploads
9. CSS engine (dev watch + build) + frontend + admin UI
10. SSR-on-demand + perf-budget enforcement