AkurAI-Build

public
Latest change 5ecb45c5915e3bb122d43dbaf90b4eba39484c7d - organize native templates and reserve runtime routes by Olafur Bui
# Architect Review 9/10 — Performance & Scalability (bunfork)

Scope: `src/server.rs`, `src/db.rs`, `Cargo.toml`, `src/main.rs`, README bounds. Read-only review; no files modified.

> Status: this review is historical. The template recompilation finding was
> fixed by startup compilation and shared environment reuse; the current
> implementation is the source of truth.

## Verdict on the documented bounds

- **Exhaustive cosine scan is defensible.** With 10M multiply-adds max (`db.rs:187`), the dot-product work is single-digit milliseconds on any modern core; ANN would add index maintenance, recall tradeoffs, and a native dependency for no measurable win at ≤10k candidates / ≤4096 dims. The `search_gate` `Semaphore(1)` (`server.rs:61,229,928`) correctly bounds concurrent scans.
- **Full-preload static serving is sound.** The 512 MiB cap is enforced pre-bind with `checked_add` (`server.rs:294-343`), and per-request serving is zero-copy via `Bytes::clone` (`server.rs:459`). RSS ≈ artifact size; no request-time filesystem I/O. Good.

## Prioritized findings

**P0 — Production renders recompile every template on every request.** `render_native_template` (`server.rs:1031-1048`) calls `template_environment` (`server.rs:1354-1366`), which copies (`source.to_string()`) and recompiles *all* shared templates plus the page template per request — even in prod. Additionally `render_page` deep-clones the entire `PageRouter` per request in prod (`server.rs:976`). Startup compilation (`server.rs:688`) is validation-only and discarded. This caps native-page throughput at O(total template bytes) per request and contradicts README's "production pages are compiled and loaded before binding." Fix: build one `Arc<Environment<'static>>` with all templates+pages at startup; dev mode keeps the rescan path.

**P1 — Blocking DB call on the async runtime in readiness probe.** `native_ready` calls `state.database.check_ready()` synchronously (`server.rs:751`; `db.rs:145`), taking the std `Mutex` on a tokio worker thread. A search holding the connection mutex for the full scan stalls a worker; on small hosts (1–2 cores) this can stall all request processing. Wrap in `spawn_blocking` (as the vector handlers already do, `server.rs` upsert/search/delete) or use `try_lock` with a fast "busy=ready" answer.

**P1 — Single `Arc<Mutex<Connection>>` serializes all DB work.** (`db.rs:53,367`). WAL is enabled (`db.rs:507-513`) but its reader/writer concurrency is unused: searches hold the mutex through count + full scan + top-k + content refetch (`db.rs:175-268`), blocking upserts, deletes, and readiness. A second read-only connection (or a tiny 2–3 reader pool) for search/ready would remove the head-of-line blocking with minimal code.

**P1 — Native `/assets` are gzip-compressed on-the-fly per request, gzip only.** `CompressionLayer::new()` (`server.rs:241`) with only `compression-gzip` enabled (`Cargo.toml:23`) recompresses identical preloaded bytes for every non-304 hit; there is no `.br`/`.gz` negotiation for native public assets (unlike static mode, `server.rs:512-535`). Precompress public assets once at preload (reuse `negotiated_static_path` logic), and consider enabling `compression-br`/`compression-zstd` features for HTML/JSON responses.

**P2 — Per-candidate allocation and double pass in the scan loop.** `decode_embedding` (`db.rs:848-863`) allocates a `Vec<f32>` and does a finiteness pass per candidate, then the dot product does a second pass (`db.rs:215-230`) — up to 10k allocations of ≤16 KiB per search. Compute the dot product directly over `chunks_exact(4)` of the blob; skip the redundant finite check (writes already validate).

**P2 — Search availability cliff at the budget.** Once a tenant/model/dimension exceeds `work_limit`, search returns 400 permanently (`db.rs:195-200`) until rows are deleted. Documented behavior, but operationally silent — consider a metric/log at e.g. 80% of budget.

**P2 — Redundant embedding validation/normalization.** Handlers validate (`server.rs` upsert/search) and the DB layer re-validates and re-normalizes (`db.rs:157-160,183-184`) — up to three passes over 4096 floats. Trivial cost; free cleanup.

**P2 — Tokio runtime config is fine as-is.** Default multi-thread `#[tokio::main]` (`main.rs:293`); blocking vector work correctly uses `spawn_blocking`, and the semaphore bounds blocking-pool fanout. No change needed; static mode's lack of a dynamic compression fallback (`server.rs:262-266`, no CompressionLayer) is acceptable but worth a README sentence for artifacts admitted without precompressed variants.

## Residual risks

- P0 template fix must preserve dev-mode reload semantics and startup fail-closed compilation.
- Adding reader connections must respect the shared-lock/exclusive-lock protocol used by migrate/restore (`db.rs:79-100`).