AkurAI-Framework / site/content/docs/semantic-search.md

Latest change 7e388dfdbf1807a94cf3959541a06d8b3daf4fb5 - Add bearer auth support for embeddings by Ólafur Búi Ólafsson

# Semantic search

The `akurai-vector` crate provides the primitives behind `?search=<text>`:
the vector math, a byte codec for storing embeddings, and a small embedding
client that talks to any OpenAI-compatible endpoint. Like every other crate in
the framework it is **pure `std`** — its only dependency is the workspace's own
`akurai-json`.

It owns the math and the embedding fetch. It does **not** wire HTTP routes or
touch collections — the CLI drives it.

## How `?search` ranks records

1. The CLI reads the embedding endpoint and model from the environment
   (`AKURAI_EMBED_URL`, `AKURAI_EMBED_MODEL`).
2. It embeds the query string into a vector.
3. It compares that query vector against each record's stored embedding using
   **cosine similarity**.
4. It returns the **top-k** records by descending similarity.

```text
?search=text  ──▶ embed(query) ──▶ cosine vs. each stored record embedding ──▶ top-k
```

### Substring fallback

When **no endpoint is configured** (`AKURAI_EMBED_URL` is unset), the CLI skips
embeddings entirely and falls back to plain **substring search** over the
records. That fallback lives in the CLI, not in this crate — `akurai-vector` is
config-free and never assumes a host.

## Vector math

```rust
use akurai_vector::{cosine, rank};

// Cosine similarity, in [-1.0, 1.0].
let s = cosine(&[1.0, 0.0], &[1.0, 0.0]); // 1.0

// Top-k by descending cosine. Returns (id, score) pairs.
let query = vec![1.0, 0.0];
let candidates = vec![
    (10u64, vec![0.0, 1.0]), // orthogonal -> 0.0
    (20u64, vec![1.0, 0.0]), // identical  -> 1.0
];
let top = rank(&query, &candidates, 1); // [(20, 1.0)]
```

`cosine` never panics. Documented edge cases:

| Case | Result |
|------|--------|
| Identical vectors | `1.0` |
| Orthogonal vectors | `0.0` |
| Opposite vectors | `-1.0` |
| Either vector zero-norm (all zeros / empty) | `0.0` |
| Mismatched lengths | `0.0` (treated as not comparable) |

`rank` is stable: ties keep their input order. `k` larger than the candidate
count returns all candidates; `k == 0` returns an empty list.

## Byte encoding for storage

Embeddings are stored as opaque byte blobs in the B+tree. The codec is a fixed
**little-endian `f32`** layout (4 bytes per component), converted explicitly so
a vector round-trips exactly on any host — no `unsafe`, no `transmute`.

```rust
use akurai_vector::{encode, decode};

let v = vec![0.1_f32, -0.2, 0.3];
let bytes = encode(&v);            // 12 bytes, little-endian
let back  = decode(&bytes);        // Some(vec![0.1, -0.2, 0.3])
assert_eq!(back, Some(v));
```

`decode` returns `None` if the blob length isn't a multiple of 4 (i.e. not a
whole number of `f32`s), so a corrupt or short blob can never panic the search
path.

## The embedding protocol (OpenAI-compatible)

`embed` / `embed_many` POST to `<endpoint>/v1/embeddings` with the body:

```json
{ "model": "<model>", "input": "<text>" }
```

or, for a batch:

```json
{ "model": "<model>", "input": ["<text>", "<text>"] }
```

and parse the response shape:

```json
{ "data": [ { "embedding": [0.1, 0.2, ...] }, ... ] }
```

```rust
use akurai_vector::{embed, embed_many, embed_with_bearer};

let v   = embed("http://localhost:8081", "embeddinggemma", "halló heimur")?;
let vs  = embed_many("http://localhost:8081", "embeddinggemma", &["a", "b"])?;
let rv  = embed_with_bearer(
    "http://127.0.0.1:4219",
    "intfloat/multilingual-e5-small",
    "halló heimur",
    "akr_...",
)?;
# Ok::<(), akurai_vector::EmbedError>(())
```

`embed_many` returns one vector per input, in order; an empty input slice
returns an empty result without any network call.

### Plain HTTP only — no TLS

The client is a minimal HTTP/1.1 implementation over `std::net::TcpStream`. It
speaks **plain HTTP only**. std has no TLS and the framework will not pull a
crate for it — **TLS terminates at the edge** (Caddy/nginx), and these embedding
endpoints are local/edge services on plain HTTP. Passing an `https://` URL
returns a clear `EmbedError::TlsUnsupported` rather than silently failing.

The endpoint is parsed as `http://host:port` (port defaults to 80; any trailing
path is ignored — the client always posts to `/v1/embeddings`). The request
carries `Host`, `Content-Type: application/json`, an exact `Content-Length`, and
`Connection: close`, and a 30-second read/write timeout is set so the search
path fails fast instead of hanging.

### Error handling

Every failure mode maps to a distinct, non-panicking `EmbedError` variant so the
CLI can react precisely (and degrade to substring search):

| Variant | When |
|---------|------|
| `InvalidEndpoint` | Endpoint isn't `http://host[:port]` |
| `TlsUnsupported` | An `https://` URL was given |
| `Connect` | TCP connect failed (refused, DNS, host down) |
| `Timeout` | Read/write timed out |
| `Io` | Other socket I/O error |
| `HttpStatus { code, snippet }` | Server replied non-2xx |
| `ShortResponse` | Empty/truncated response, no header/body split |
| `MalformedJson` | Body wasn't valid JSON |
| `UnexpectedShape` | JSON didn't match `{"data":[{"embedding":[...]}]}` |
| `CountMismatch` | Returned a different number of embeddings than requested |

## Testing

The pure parts are factored out and fully unit-tested without ever touching the
network: `build_request`, `parse_endpoint`, `split_response`, and
`parse_embeddings_response` are checked against hand-written request/response
strings, alongside the `cosine` / `rank` math and the `encode` / `decode`
round-trip. The actual `TcpStream` round-trip (`fetch`) is the only part that is
not unit-tested, by design.