AkurAI-Framework / site/content/docs/agent-os-ui.md

Latest change 631417472adebacf9be16c712af89ca5d8a25c28 - Add RustAgent agent OS UI contract by Ólafur Búi Ólafsson

# AkurAI-RustAgent agent OS UI

Status: ⬜ planned integration contract.

The AkurAI-RustAgent web surface must be a tenant-scoped agent operating system,
not a plain chat box. It should preserve the same interaction semantics an operator
gets from a mature agent runtime: channels, tool calls, confirmations, questions,
edit previews, artifacts, durable work queues, and receipts back to the agent.

The framework already ships the native-ESM assistant-ui + tool-ui substrate needed
for that UI. The remaining work is an adapter that turns RustAgent/Hermes-style
events into framework component payloads and sends user decisions back as receipts.

## Required surfaces

- **Workspace shell**: tenant-aware sidebar for threads, projects, notes, memory,
  passvault, schedules, kanban, and curator jobs.
- **Run timeline**: ordered channel stream for user messages, assistant responses,
  reasoning, tool calls, command output, errors, and final answers.
- **Context pane**: relevant notes, memory entries, files, artifacts, environment,
  and active task state for the selected run.
- **Approval lane**: confirmations and sensitive actions stay visible until the
  user approves, rejects, edits, or cancels.
- **Artifact/preview pane**: code diffs, rendered files, terminal output, tables,
  charts, media, and generated previews.

## Channel contract

The adapter should keep channel identity explicit. A run event should include:

```json
{
  "run_id": "run_123",
  "sequence": 42,
  "channel": "commentary",
  "kind": "tool_call",
  "tool_call_id": "call_abc",
  "payload": {}
}
```

Recommended channels:

| Channel | UI treatment |
|---|---|
| `analysis` | Collapsible reasoning block. Hidden by default unless the run policy permits display. |
| `commentary` | Main visible agent stream, including progress updates and tool-call narration. |
| `final` | Pinned final answer for the run. |
| `tool_call` | Tool card with arguments, status, timing, and output placeholder. |
| `tool_result` | Result card matched to the originating tool call. |
| `approval` | Blocking approval card with approve/reject receipts. |
| `question` | Blocking input card: option list, question flow, text field, slider, or preferences panel. |
| `edit` | Code diff or file preview with accept/reject/apply receipts. |
| `artifact` | Rendered artifact, media, table, chart, or link preview. |
| `system` | Small timeline note for policy, environment, or routing metadata. |
| `error` | Error card with retry/recover affordances where available. |

## Hermes-style asks

The UI must treat agent asks as blocking structured requests, not prose. At
minimum, support these request/response pairs:

| Request event | Payload | Response method | Response payload |
|---|---|---|---|
| `clarify.request` | `{ question, choices?, request_id }` | `clarify.respond` | `{ request_id, answer }` |
| `approval.request` | `{ command, description, pattern_key?, pattern_keys?, allow_permanent?, request_id? }` | `approval.respond` | `{ session_id, choice, all? }` |
| `sudo.request` | `{ prompt?, request_id }` | `sudo.respond` | `{ request_id, password }` |
| `secret.request` | `{ prompt, env_var, metadata?, request_id }` | `secret.respond` | `{ request_id, value }` |
| `terminal.read.request` | `{ prompt?, request_id }` | `terminal.read.respond` | `{ request_id, text }` |

Approval choices should keep Hermes semantics: `once`, `session`, `always`, or
`deny`. If `allow_permanent` is false, do not show or persist `always`.
Clarification choices should stay short and bounded; open-ended clarification is
represented by omitting `choices`.

## Component mapping

Use the shipped tool-ui components before creating new UI:

| Agent event | Framework component |
|---|---|
| Multi-step plan or kanban progress | `plan`, `progress_tracker` |
| `approval.request` for deploy, delete, edit, or external action | `approval_card` or `genericApprovalComponent` |
| `clarify.request` with choices | `option_list` |
| `clarify.request` without choices or `terminal.read.request` | `question_flow` with a text question |
| Ask several questions before proceeding | `question_flow` |
| Ask for preferences or run settings | `preferences_panel` |
| Ask for a numeric parameter | `parameter_slider` |
| `sudo.request` | Dedicated password prompt with no echo and no persistent value |
| `secret.request` | Dedicated secret prompt with env var and redacted metadata |
| Show command/tool output | `terminal`, `code_block` |
| `tool.complete.inline_diff`, rollback diffs, or file edits | `code_diff` |
| Show structured records | `data_table`, `chart`, `stats_display` |
| Show generated or discovered media | `image`, `image_gallery`, `audio`, `video` |
| Show link/citation context | `link_preview`, `citation` |

If no component matches, the runtime must fall back to readable JSON rather than
dropping the event.

## Receipts back to the agent

Interactive components must reply through the existing receipt shape:

```json
{
  "channel": "tool-result",
  "receipt": {
    "toolCallId": "call_abc",
    "component": "approval_card",
    "action": "approve",
    "data": { "approved": true },
    "ts": 1782487573184
  }
}
```

For RustAgent, that receipt should be forwarded to the run/gateway as the answer to
the pending agent ask. The UI should not convert approvals into plain chat text; it
must preserve the target `toolCallId`, selected action, and typed data.

The receipt bridge maps component receipts to the matching response method:

| Component receipt | Agent response |
|---|---|
| `approval_card` approved once/session/always/deny | `approval.respond` |
| `option_list` or `question_flow` for clarification | `clarify.respond` |
| secret prompt value | `secret.respond` |
| sudo prompt value | `sudo.respond` |
| terminal read text | `terminal.read.respond` |
| edit preview accept/reject | edit-specific response or tool-result receipt keyed by `toolCallId` |

## Implementation slices

1. **Event adapter**: normalize RustAgent/Hermes gateway events into the channel
   contract above, preserving sequence, run id, tool call id, and tenant id.
2. **Workspace shell**: compose ThreadList, branch picker, reasoning, suggestions,
   tool-call runtime, and the artifact components into an agent-OS layout.
3. **Receipt bridge**: route `toolui:receipt` and component confirmations back to
   RustAgent as structured answers, then update the pending timeline item.
4. **Durable state panes**: wire memory, notes, passvault, kanban, cron, and curator
   records as first-class sidebar/context objects.
5. **Hermes parity checks**: build fixtures for confirmations, questions, tool
   calls, edits, artifacts, channel ordering, and recovery from disconnected runs.