AkurAI-Framework / site/content/docs/collections.md

Latest change 11133e70d7ecc0416eabf8ca4bb82fd3dd873147 - feat(collections): add akurai-collections auto-generated-API engine by Ólafur Búi Ólafsson

# Collections — the auto-generated API engine

Declare a **collection** — a name plus a set of typed fields — and the framework
gives you a full REST resource over the embedded database: create, read, list,
update, delete, and a baseline search. No hand-written endpoints, no ORM, no
external dependencies. The engine is the `akurai-collections` crate.

## The schema model

A collection is a `name` and an ordered list of `fields`. Each field declares:

| Property   | Meaning                                                              |
|------------|---------------------------------------------------------------------|
| `name`     | The field's key in the JSON record. Must be unique in the collection. |
| `kind`     | One of `text`, `int`, `float`, `bool`.                              |
| `required` | When true, the field must be present and non-null on create.        |
| `embed`    | Marks a `text` field for **later** semantic indexing (see below).   |

The CLI builds these schemas from your `collections.toml` at startup. The engine
itself is decoupled from TOML — it consumes plain schema values.

### Field types

- **`text`** — a UTF-8 string. Only `text` fields participate in search.
- **`int`** — a 64-bit signed integer.
- **`float`** — a 64-bit float. Integer inputs are accepted and coerced to float.
- **`bool`** — `true` / `false`.

## Reserved fields the engine owns

Every stored record carries two engine-assigned keys that your input may **not**
set:

- **`id`** — an auto-incrementing `u64`, unique per collection, never reused
  (even after deletes).
- **`created`** — the creation time in unix seconds.

A stored record is a JSON object whose first two keys are always `id` and
`created`, followed by the schema fields you supplied, in declaration order.
Optional fields you omit are left out of the record entirely (not stored as
`null`).

## Record key layout

Records live in the B+tree under a per-collection prefix:

```text
coll:<name>:<id>     <id> is a big-endian u64   →  the record (JSON bytes)
coll:<name>:_seq     the auto-increment counter (big-endian u64, last id used)
```

Because ids are fixed 8-byte big-endian, they sort numerically, so a range scan
over the prefix returns records in insertion order. The `_seq` counter shares
the prefix but has a different key length, so scans filter on the exact record
key length and never confuse the two. Each collection has its own counter, so
ids in `posts` and `tags` are independent.

## The REST endpoints

For a collection named `posts`, the framework exposes:

| Method & path                              | Action                          |
|--------------------------------------------|---------------------------------|
| `GET /api/collections/posts/records`       | List records (newest first).    |
| `POST /api/collections/posts/records`      | Create a record from a JSON body. |
| `GET /api/collections/posts/records/<id>`  | Fetch one record by id.         |
| `PATCH /api/collections/posts/records/<id>`| Partially update a record.      |
| `DELETE /api/collections/posts/records/<id>`| Delete a record.               |
| `GET /api/collections/posts/records?search=<query>` | Substring search. |

`list` and `search` return newest-first (highest id first) and accept an
optional `limit`.

These same descriptors are emitted by the engine's manifest and merged into the
framework-wide `/api/_meta` document, so clients can discover every collection,
its fields, and its endpoints at runtime.

## Validation rules

The engine never panics on bad input — it returns a validation error.

**On create:**

- Every `required` field must be present and non-null.
- Every supplied value must match its field's declared type (`int` is accepted
  where a `float` is expected and coerced).
- Unknown fields (not in the schema) are rejected.
- The reserved keys `id` and `created` may not be supplied.
- The body must be a JSON object.

**On update (PATCH):** the patch is partial — only the keys you send change.

- Each changed field is re-validated against the schema (type checked).
- Unknown fields and the reserved keys are rejected.
- Setting a `required` field to `null` is rejected.
- Setting an *optional* field to `null` clears (removes) it from the record.
- Updating a non-existent id returns "not found" (no error).

## Search

`?search=<query>` is an **honest baseline**: a case-insensitive substring match
across the collection's `text` fields. A record matches if any of its `text`
fields contains the query. Non-text fields are never searched. An empty query
matches everything. Results are newest-first and honor `limit`.

This is a deliberate, predictable baseline — **semantic ranking is layered on
top** by the framework's vector layer, not by this engine.

## The `embed` flag

Setting `embed` on a `text` field marks it for semantic indexing. The
collections engine **stores and exposes** the flag (it appears in `/api/_meta`)
but does not embed anything itself. The CLI's vector crate reads the flag and
builds the semantic index on top of the records this engine stores. So `embed`
is a contract between your schema and the semantic layer — the substring
`?search=` above works regardless.