AkurAI-Framework / site/content/docs/generative-ui.md

Latest change 631417472adebacf9be16c712af89ca5d8a25c28 - Add RustAgent agent OS UI contract by Ólafur Búi Ólafsson

# Generative UI runtime

The engine behind **tool-ui**: it maps a tool-call payload to a DOM component and
renders it, so the assistant can return *rich interactive UI* instead of plain
text. Pure native ES modules, zero dependencies, no build step. It lives under
`/ai/runtime/` and you activate the whole thing by importing one file:

```js
import Runtime from "/ai/runtime/index.js";
```

Importing `index.js` also wires the **chat-core bridge** (it listens for
`chat:toolcall` events — see below). Every module logs `[toolui]` / `[runtime]`
breadcrumbs to the console.

See it live at **[/ai/runtime-demo](/ai/runtime-demo)**.

## The component contract

A generative-UI component is a plain object. **Pin this shape** — the dispatcher
and every component author conform to it:

```js
const def = {
  name:     "weather-card",                 // unique id
  category: "data",                         // grouping for catalogs
  title:    "Weather card",                 // human label
  match:    (payload) => payload?.type === "weather",   // does this def render it?
  render:   (payload, ctx) => HTMLElement,  // build & return the DOM node
  example:  { type: "weather", city: "Akureyri", temp: 4 }, // sample payload
};
```

| Field | Type | Purpose |
|---|---|---|
| `name` | string (required) | Unique id. Re-registering a name replaces it in place. |
| `category` | string | Grouping for component catalogs. Defaults to `"uncategorized"`. |
| `title` | string | Human-readable label. Defaults to `name`. |
| `match(payload)` | `(any) => boolean` (required) | Return `true` if this def should render the payload. |
| `render(payload, ctx)` | `(any, ctx) => HTMLElement` (required) | Build and return the DOM node. |
| `example` | any | Sample payload for docs, demos, and tests. |

### The `ctx` shape

Every `render` receives a `ctx`:

```js
ctx = {
  sendReceipt(obj),       // interactive components reply to the assistant
  onAction(name, data),   // generic action callback (clicks etc.)
  theme,                  // "dark" (default) | "light" | author value
};
```

`ctx` is always normalized before it reaches a component, so you can call
`ctx.sendReceipt(...)` and `ctx.onAction(...)` without guarding for `undefined`.

## Registering and dispatching

```js
import { registerToolComponent, registerAll, renderToolPayload } from "/ai/runtime/index.js";

registerToolComponent(def);          // one
registerAll([defA, defB]);           // many → returns count registered

const node = renderToolPayload(payload, ctx);   // → HTMLElement
container.appendChild(node);
```

`renderToolPayload` walks registered defs **in registration order** and renders
the **first** whose `match(payload)` returns `true`. Register specific components
before generic ones.

### Fail-safe fallback

If **no** def matches — or a component's `match`/`render` throws — the dispatcher
never crashes. It returns a `.card` containing the pretty-printed JSON payload in
a `.code-block`. Your UI degrades to readable JSON instead of breaking.

## Receipts — replying to the assistant

When an interactive component (an approval, a form) resolves, it calls
`ctx.sendReceipt(obj)`. The receipt travels back to the assistant as the result
of the tool call that produced the component.

`sendReceipt` does **both**, always:

1. dispatches a `toolui:receipt` DOM `CustomEvent` (detail = the receipt), so the
   chat core or tests can observe it with zero network, and
2. attempts a `POST /api/assistant` on the tool-result channel, swallowing
   network errors gracefully (logged, never thrown).

### Receipt POST shape

```http
POST /api/assistant
Content-Type: application/json

{
  "channel": "tool-result",
  "receipt": {
    "toolCallId": "call_abc",
    "component":  "weather-card",
    "action":     "approve",
    "data":       { "any": true },
    "ts":         1782487573184
  }
}
```

`sendReceipt(obj)` returns a `Promise<{ ok, status, posted }>` and **never
rejects** — components don't have to `try/catch`. It accepts loose input
(`tool_call_id`, `value`, `name`) and normalizes it into the envelope above.

## Inline approvals

`renderApproval(opts, ctx)` is the human-in-the-loop plumbing — a generic
Approve/Reject prompt. (A polished Approval Card *component* can be registered
separately; this is the runtime underneath it.)

```js
import { renderApproval } from "/ai/runtime/index.js";

const { element, promise } = renderApproval(
  { title: "Delete 3 records?", message: "This can't be undone.", toolCallId: "call_1" },
  ctx
);
container.appendChild(element);
const { approved } = await promise;   // resolves after the user chooses
```

Choosing an option resolves the promise **and** calls `ctx.sendReceipt({ action:
"approve" | "reject", data: { approved } })`.

A `genericApprovalComponent` def is exported too. It is **not** auto-registered
(first-match-wins would let it shadow a purpose-built card) — register it
explicitly, ideally last:

```js
import { registerToolComponent, genericApprovalComponent } from "/ai/runtime/index.js";
registerToolComponent(genericApprovalComponent);   // matches { type: "approval", ... }
```

## The chat-core bridge

Importing `index.js` registers a listener for `chat:toolcall`. The chat core
dispatches a tool call; the runtime renders it:

```js
document.dispatchEvent(new CustomEvent("chat:toolcall", {
  detail: {
    call,              // original assistant tool call, when available
    payload,           // the tool-call payload to render
    target,            // (optional) element to append the rendered node into
    ctx,               // (optional) partial ctx, merged over the default
    toolCallId,        // (optional) threaded into receipts
  },
}));
```

The runtime renders with `renderToolPayload`, appends the node to `detail.target`
(if given), and dispatches `toolui:rendered` with
`detail = { element, payload, toolCallId }`. The default `ctx` routes
`sendReceipt` through the receipts transport, so interactive components reach the
assistant out of the box.

## Conversation primitives

Small, composable building blocks. Each returns an `element` plus methods, and
emits a `CustomEvent` on change.

### ThreadList — `createThreadList(opts)`

Switch among conversations. In-memory, mirrored to `localStorage`. Emits
`thread:select` (detail = the thread).

```js
import { createThreadList } from "/ai/runtime/index.js";

const threads = createThreadList({
  threads: [{ id: "t1", title: "Weather" }],
  storageKey: "toolui:threads",   // null disables persistence
  onSelect: (t) => console.log(t.id),
});
sidebar.appendChild(threads.element);
// API: add(thread), remove(id), select(id), rename(id, title), list(), getSelected()
```

### Branch picker — `createBranchPicker(opts)`

Navigate alternate message versions (`‹ k/N ›`). Emits `branch:change`
(detail = `{ index, total }`).

```js
import { createBranchPicker } from "/ai/runtime/index.js";

const branch = createBranchPicker({ total: 3, index: 0, onChange: (i, n) => {} });
msg.appendChild(branch.element);
// API: setIndex(i), next(), prev(), getIndex(), getTotal(), setTotal(n)
```

### Reasoning — `createReasoning(opts)`

A collapsible thinking block (native `<details>`), closed by default. Emits
`reasoning:toggle` (detail = `{ open }`).

```js
import { createReasoning } from "/ai/runtime/index.js";

const r = createReasoning({ summary: "Show reasoning", text: "Step 1…\nStep 2…" });
turn.appendChild(r.element);
// API: setText(text), setOpen(bool), isOpen()
```

### Suggestions — `createSuggestions(items, opts)`

Tappable chips that fill (and optionally submit) the composer. Emits
`suggestion:pick` (detail = `{ text, submit }`).

```js
import { createSuggestions } from "/ai/runtime/index.js";

const sug = createSuggestions(
  ["Tell me more", { text: "Send now", submit: true }],
  { composer: inputEl, form: composerForm }   // composer/form optional
);
container.appendChild(sug.element);
// API: setItems(items)
```

## Events at a glance

| Event | Emitted by | `detail` |
|---|---|---|
| `chat:toolcall` | chat core → runtime (you dispatch) | `{ call?, payload, target?, ctx?, toolCallId? }` |
| `toolui:rendered` | runtime, after rendering | `{ element, payload, toolCallId }` |
| `toolui:receipt` | `sendReceipt` | the normalized receipt |
| `thread:select` | ThreadList | the selected thread |
| `branch:change` | Branch picker | `{ index, total }` |
| `reasoning:toggle` | Reasoning block | `{ open }` |
| `suggestion:pick` | Suggestion chips | `{ text, submit }` |

## Design notes

- **No build step.** Every file is a hand-written ES module the browser loads
  directly. The design system (`/styles.css`) supplies all classes — components
  reuse `.card`, `.btn`, `.code-block`, `.badge` rather than ship their own CSS.
- **Correct by construction.** The dispatcher can only fail safe; receipts only
  fail silent; pickers clamp their indices. The UI cannot crash on bad input.
- **One import to activate.** `import "/ai/runtime/index.js"` registers the bridge
  and exposes `window.ToolUiRuntime` for ad-hoc use.