AkurAI-Framework / site/content/docs/ai-components.md

Latest change bd7e7af40322268b3b891951645f1059a5220975 - docs: AI components overview (assistant-ui + tool-ui 1:1 port plan) + AI Components batch track (AB1-AB4); fix stale binary-size stat (~1.1 MB) by Ólafur Búi Ólafsson

# AI components

A built-in **AI chat UI** and a **tool-result component library**, ported into the
framework's no-build, native-ESM frontend. The goal is a production-grade assistant
interface — streaming chat, generative UI, and rich tool-call visualizations — with
zero bundler, zero runtime dependencies, and no React.

> **Provenance.** The component set and patterns are a 1:1 port (in spirit, not in
> code) of two excellent React/TypeScript libraries:
> [assistant-ui](https://github.com/assistant-ui/assistant-ui) (composable chat
> primitives) and [assistant-ui/tool-ui](https://github.com/assistant-ui/tool-ui)
> (tool-payload → UI). We reimplement their components and UX in vanilla ES modules
> on top of `signals.js`, the design system, and `akurai-markdown` — no React, no
> Tailwind runtime, no Zod. Credit to the assistant-ui authors for the design.

## Why a port, not a dependency
assistant-ui is React + a bundler + Tailwind + Zod. This framework is the opposite:
hand-written ES modules the browser loads directly, a from-scratch SSR template
engine, and a pure-`std` server. So we port the **components and behaviours** — the
Radix-style composable primitives, the streaming UX, the tool-UI mapping — into the
framework's idiom. You get the same capabilities with none of the build chain.

## Chat primitives (from assistant-ui)
Composable, Radix-style building blocks — you assemble a thread rather than drop in a
monolith:

| Primitive | What it does |
|---|---|
| **Thread** | The conversation container — scrollback, auto-scroll, scroll-to-bottom. |
| **Message** | Renders one message (user / assistant / system), Markdown + code highlighting via `akurai-markdown`. |
| **Composer** | The input: send, multiline, keyboard shortcuts, attachments, optional voice dictation. |
| **ActionBar** | Per-message actions — copy, retry, edit, branch. |
| **ThreadList** | Switch between multiple conversations. |
| **Branch picker** | Navigate alternate message branches (retries/edits). |
| **Reasoning** | Collapsible "thinking" / reasoning display for assistant turns. |
| **Suggestions** | Tappable follow-up prompts. |

Built-in UX: **streaming** (token-by-token over SSE), auto-scroll, retries,
attachments, Markdown, syntax-highlighted code, keyboard shortcuts, and
accessibility.

## Generative UI & the tool-call runtime
The assistant can render **custom UI for tool calls and structured JSON**, not just
text. A small client runtime maps a tool result to a component:

1. A tool call/result arrives (over the stream).
2. The runtime matches its payload against a registered component's **schema**
   (the framework's own lightweight validation — no Zod).
3. On a match, the component renders inline in the message; on a mismatch it **fails
   safely** (falls back to a JSON/code view).
4. User interactions (an approval, a form submit) persist as **receipts** that flow
   back to the assistant as tool output.

Also: **inline human approvals** (confirm an action before it runs) and **safe
frontend actions** (expose controlled capabilities to the model).

## Tool-result components (from tool-ui)
30+ components across six categories, each driven by a typed payload:

- **Progress** — Plan, Progress Tracker.
- **Input** — Option List, Parameter Slider, Preferences Panel, Question Flow.
- **Display** — Citation, Geo Map, Item Carousel, Link Preview, Stats Display,
  Terminal, Weather Widget.
- **Artifacts** — Chart, Code Block, Code Diff, Data Table, Message Draft, and social
  post cards (X, LinkedIn, Instagram).
- **Confirmation** — Approval Card, Order Summary.
- **Media** — Audio, Image, Image Gallery, Video.

Every component ships with example/preset data so a tool author can preview the
mapping before wiring a real tool.

## Framework support (added for this)
A chat UI needs a backend. Where the framework was missing a piece, we add it — in
the same pure-`std`, zero-dependency spirit:

- **LLM client** (`crates/llm`, new) — a plain-HTTP client for any OpenAI-compatible
  `/v1/chat/completions` endpoint, including **token streaming** (SSE), mirroring how
  `akurai-vector` talks to an embeddings endpoint. Endpoint + model come from env
  (`AKURAI_LLM_URL` / `AKURAI_LLM_MODEL`); HTTPS is rejected (no TLS in `std` — point
  it at a local/edge endpoint).
- **Streaming assistant endpoint** — `POST /api/assistant` proxies a conversation to
  the configured model and **streams tokens back over SSE**, so `Thread` renders
  responses live. Degrades to a clear error when no LLM endpoint is configured.
- **Tool-call protocol** — the assistant endpoint relays model **tool calls** to the
  client and accepts tool **results/receipts** back, so generative UI and approvals
  round-trip.
- **Payload schemas** (`crates/schema`, new) — a lightweight JSON-shape validator
  (no Zod) used both server-side and exposed to the client, so a tool result is
  matched to the right component and rejected cleanly when it doesn't fit.
- **Attachments** reuse the existing content-addressed blob store; **voice dictation**
  uses the browser's native Web Speech API (no server STT).

## Status
This is an in-progress port, tracked as the **AI Components track** in the repo's
`BATCHES.md`. Components land incrementally with their docs; this page is the
overview and will link each component's reference as it ships. Charts, maps, and
media are rendered with native browser APIs (SVG, `<canvas>`, `<video>`/`<audio>`)
— no charting or mapping library is bundled.