Menu
Public repository / c2f43d4988686a5b5142bb6dd4255f9b473792dc
AkurAI-Notes / .plan/epics/E05-embeddings-and-semantic-search.md
Latest change f1d9b8cc6ff48113d891a2d95fd79b2c49f1f4ff - Initial plan and .plan directory by Claude Code
# E05 — On-device embeddings + semantic search
## Goal
A `GET /api/search?q=<text>` route that:
1. embeds the query via a local OpenAI-compatible /v1/embeddings server,
2. iterates `note_emb:<id>` keys in the B+tree, decodes each f32 vec, and
3. cosine-ranks the top-k (default 20) notes by similarity, owner-scoped.
4. Falls back to substring search when `AKURAI_NOTES_EMBED_URL` is unset (matches
Framework's `collections_api.rs` fallback discipline — principle #5 +
progressive enhancement).
## Reference sources
- `~/Projects/AkurAI-Framework/crates/vector/src/lib.rs` — the `embed`,
`embed_many`, `encode`, `decode`, `cosine`, `rank` signatures.
- `~/Projects/AkurAI-Framework/crates/vector/src/client.rs` —
HTTP/1.1 to `/v1/embeddings` shape (parsed JSON, OpenAI-compatible).
- `~/Projects/AkurAI-Framework/crates/cli/src/collections_api.rs` — the live
demo's `?search` handler that already wires all of this together. Notes
mostly ports it.
- `~/Projects/AkurAI-Framework/AGENTS_DATA.md` — the B+tree range-scan idiom.
## Architecture
```
┌──────────────────────────────────────────────┐
│ notes-api::handlers::search │
│ GET /api/search?q=intent&k=20 │
└──────┬───────────────────────┬────────────────┘
│ │
AKURAI_NOTES_EMBED_URL α α (global switch)
│ │
▼ ▼
(embed path) (substring path)
│ │
embed(q, url, model) substring match over
→ Vec<f32> `note:*` keys' JSON
│ title/body
▼ │
iter note_emb:* (range_prefix) │
for each: decode(v) → Vec<f32> │
for each: cosine(qv, docv) → f32 │
rank(scored, k) │
│ │
└──────┬────────────────┘
▼
ownerfilter on ownerSub ==
session.user.sub
▼
resolve note:<id> for top-k
▼
JSON response { query, results: [{ id, title, score, snippet }] }
```
## Steps
### 1. Create `crates/notes-api/src/embed.rs` (port the math layer from `CODE-PATTERNS.md §7`)
Implement two functions on top of `akurai-vector`:
```rust
use akurai_storage::BTree;
use akurai_vector::{embed, encode, decode, cosine, rank, EmbedError};
pub struct EmbedStore<'a> {
pub db: &'a mut BTree,
}
impl<'a> EmbedStore<'a> {
/// Persist the embedding for a note. Called after create/update of note body.
pub fn store(&mut self, note_id: i64, vec: &[f32]) -> Result<(), String> {
let key = format!("note_emb:{}", note_id);
let bytes = encode(vec);
self.db.insert(key.as_bytes(), &bytes).map_err(|e| e.to_string())?;
self.db.commit().map_err(|e| e.to_string())?;
Ok(())
}
/// Rank the user's notes against a query vector. Invalid embeddings are skipped.
pub fn rank_notes(&self, qv: &[f32], owner_sub: &str, k: usize) -> Vec<(i64, f32)> {
let mut scored: Vec<(i64, f32)> = Vec::new();
for (key, val) in self.db.range_prefix(b"note_emb:") {
let id_str = std::str::from_utf8(&key[b"note_emb:".len()..]).ok();
let id: i64 = match id_str.and_then(|s| s.parse().ok()) { Some(i) => i, None => continue };
if !self.note_is_owned_by(id, owner_sub) { continue; }
let doc = match decode(val) { Ok(v) => v, Err(_) => continue };
scored.push((id, cosine(qv, &doc)));
}
rank(scored, k.unwrap_or(20)) // akurai-vector::rank sorts desc + truncates
}
fn note_is_owned_by(&self, id: i64, want: &str) -> bool {
let key = format!("note:{}", id);
match self.db.get(key.as_bytes()) {
Ok(Some(bytes)) => {
if let Ok(v) = akurai_json::parse(&String::from_utf8_lossy(&bytes)) {
v.get("ownerSub").and_then(|x| x.as_str()) == Some(want)
} else { false }
}
_ => false,
}
}
}
/// Wrapper used by the search handler.
pub fn embed_query(text: &str, url: &str, model: &str) -> Result<Vec<f32>, EmbedError> {
embed(url, model, text)
}
```
`range_prefix` on `akurai-storage::BTree`: if the type doesn't already have a
`range_prefix(prefix)` method, construct an upper bound by appending `0xff`
to the prefix (the CRM `router_setup.rs::upper_bound` helper). Verify the
storage API by grepping `~/Projects/AkurAI-Framework/crates/storage/src/lib.rs`
for `pub fn` — don't guess.
### 2. Write `crates/notes-api/src/handlers/search.rs`
```rust
use crate::embed::{embed_query, EmbedStore};
use crate::router_setup::{json_response, error_response};
use akurai_http::{Request, Response};
use akurai_json::{parse, Value};
use std::sync::{Arc, Mutex};
/// GET /api/search?q=<text>&k=<n>
/// Returns: { "query": "...", "mode": "semantic"|"substring", "results": [...] }
pub fn search_route(state: Arc<Mutex<crate::router_setup::NotesState>>) -> RouteHandler {
Box::new(move |req: &Request| -> Response {
let user = match require_session(&state, req) { Some(u) => u, None => return error_response(401, "unauthorized") };
let q = req.query.get("q").unwrap_or_default();
let k: usize = req.query.get("k").and_then(|s| s.parse().ok()).unwrap_or(20);
if q.is_empty() { return json_response(Value::Object(vec![
("query".into(), Value::Str("".into())),
("mode".into(), Value::Str("none".into())),
("results".into(), Value::Array(vec![])),
])); }
let mut s = state.lock().unwrap();
let sem_mode = s.embed_url.is_some();
if sem_mode {
let url = s.embed_url.as_ref().unwrap();
let model = s.embed_model.as_deref().unwrap_or("qwen3-embedding-0.6b");
match embed_query(q, url, model) {
Ok(qv) => {
let store = EmbedStore { db: &mut s.db };
let ranked = store.rank_notes(&qv, &user.sub, k);
let results: Vec<Value> = ranked.iter().map(|(id, score)| {
let note = fetch_note(&s.db, *id);
note_to_searchresult(*id, score, note)
}).collect();
return json_response(Value::Object(vec![
("query".into(), Value::Str(q.into())),
("mode".into(), Value::Str("semantic".into())),
("results".into(), Value::Array(results)),
]));
}
Err(e) => { /* fall through to substring */ eprintln!("embed failed: {e}"); }
}
}
// substring path
let results = substring_search(&s.db, q, &user.sub, k);
json_response(Value::Object(vec![
("query".into(), Value::Str(q.into())),
("mode".into(), Value::Str("substring".into())),
("results".into(), Value::Array(results)),
]))
})
}
```
A snippet (first ~180 chars of body surrounding the first match) is built
inside `note_to_searchresult`. See Oblivion's snippet rendering in
`.references/oblivion/…` for a UX-matched approach (do not copy code — take
shape).
### 3. Trigger re-embedding on note update
`notes-api/src/handlers/crud.rs` `update_route` (written in E04 with a
`// TODO: see E05` placeholder): after the underlying B+tree put, if the
record's `body` field changed, regenerate the embedding for that note and
store it via `EmbedStore::store`. The diff-check is cheap: read the old
record before applying the PUT, compare `body`. If `embed_url` is empty,
skip silently — updates work without embeddings.
### 4. Re-embed on `create` (insert via Framework auto-API vs explicit handler)
If the Framework's auto-API (`collections_api.rs`) already handles embed-on-
insert via `embed = true` in `collections.toml`, verify by reading that
file's insert handler. **If it does:** Notes doesn't need its own
`create_route` body — it just declares `embed=true` and lets the Framework do
it. **If it doesn't:** Notes' `create_route` is a fork with the same
insert + EmbedStore::store pattern.
Either way, the test in §6 below confirms an embedding exists for every
inserted note.
### 5. Provision the on-device embedding server (deploy-side setup in E07)
This is deploy-side setup — E07 installs the `/opt/akurai-notes-embed/`
model file, the `systemd` unit for `llama-server` (or chosen runner), and
sets `AKURAI_NOTES_EMBED_URL=http://127.0.0.1:8081/v1/embeddings` in
`/etc/akurai-notes/notes.env`. **For E05 itself,** assume the embed server
is running on `127.0.0.1:8081` already (the seed will create a systemd unit
for it, or just run it foregrounded during development:
```bash
# Start a test embed server on titan-style box (uses existing llama.cpp on this host):
# OR run a python one-off for development only (NOT in production):
pip install --user -q fastembed sentence-transformers 2>&1 | tail -3
# Wrap it as an OpenAI-compatible stub on 127.0.0.1:8081 (skeleton):
python -c "
from fastembed import TextEmbedding
from http.server import BaseHTTPRequestHandler, HTTPServer
import json
m = TextEmbedding('Qwen/Qwen3-Embedding-0.6B') # pick smallest CPU model
class H(BaseHTTPRequestHandler):
def do_POST(self):
if self.path != '/v1/embeddings': self.send_error(404); return
L = json.loads(self.rfile.read(int(self.headers.get('Content-Length', 0))))
toks = L['input'] if isinstance(L['input'], list) else [L['input']]
embs = [list(map(float, e)) for e in m.embed(toks)]
body = json.dumps({'object':'list','data':[{'object':'embedding','index':i,'embedding':e} for i,e in enumerate(embs)], 'model': L.get('model','')}).encode()
self.send_response(200); self.send_header('Content-Type','application/json'); self.send_header('Content-Length',len(body)); self.end_headers(); self.wfile.write(body)
def log_message(self, *a): pass # quiet
HTTPServer(('127.0.0.1', 8081), H).serve_forever()
" &
curl -sS http://127.0.0.1:8081/v1/embeddings \
-H 'Content-Type: application/json' \
-d '{"model":"qwen3","input":"hello world"}' | jq .data[0].embedding | head
```
If `fastembed` is unavailable on this machine (it pulled a tiny model bundle
into `~/.cache/fastembed`), instead point Notes at the AkurAI-Router on
`titan` (the lane with `llama-server` already running) — set
`AKURAI_NOTES_EMBED_URL=http://titan.local:8080/v1/embeddings` while
developing locally. **Either way the Notes binary doesn't care — it speaks
the OpenAI protocol and points at `AKURAI_NOTES_EMBED_URL`.**
### 6. Tests
`crates/notes-api/tests/embed_test.rs`:
```rust
#[test]
fn search_returns_semantic_results_when_embed_url_set() { … }
#[test]
fn search_falls_back_to_substring_when_embed_url_empty() { … }
#[test]
fn create_note_stores_embedding() { … } // requires embed test server up
#[test]
fn update_note_reembeds_only_when_body_changes() { … }
#[test]
fn search_is_owner_scoped() { … }
```
The CI gate is "the substring tests pass" (embed server may not run in CI).
The embed tests are tagged `#[ignore]` and run manually with `cargo test --
--ignored` once an embed server is up.
### 7. Update notes.js (frontend) to use the search route
The frontend already has a search box (`site/frontend/notes.js` written in
E02's UI shape; verify there's a `<input data-model="searchQuery"
@input="search" />`). Add a debounced `fetch('/api/search?q=…')` call that
renders the results list below, showing title + snippet + score %
(indicator). The `spa.js` router doesn't care — the search is in-page.
## Files touched
```
crates/notes-api/src/embed.rs ← NEW
crates/notes-api/src/handlers/search.rs ← NEW
crates/notes-api/src/handlers/crud.rs ← edited (re-embed on update)
crates/notes-api/src/router_setup.rs ← edited (mount /api/search)
crates/notes-api/tests/embed_test.rs ← NEW
site/frontend/notes.js ← search-result renderer
site/backend/collections.toml ← no change (embed=true already from E04)
site/backend/routes.json ← edited (/api/search → handler)
```
## Acceptance
```bash
cargo test --workspace
cargo test --workspace -- --ignored # with embed server running locally
AKURAI_NOTES_DB=/tmp/e5-test.db akurai-notes seed
akurai-notes serve --db /tmp/e5-test.db --port 9811 \
--embed-url http://127.0.0.1:8081/v1/embeddings \
--embed-model qwen3-embedding-0.6b &
sleep 1
# Substring search (embed URL set but degenerate query):
curl -fsS 'http://127.0.0.1:9811/api/search?q=meeting' | jq .mode # → "semantic"
# Empty query:
curl -fsS 'http://127.0.0.1:9811/api/search?q=' | jq .mode # → "none"
# Without embed URL → substring:
akurai-notes serve --db /tmp/e5-test.db --port 9812 &
curl -fsS 'http://127.0.0.1:9812/api/search?q=meeting' | jq .mode # → "substring"
kill %1 %2
```
## Time
≈ 25 minutes.
## Depends on
- E01 (workspace scaffold + CLI)
- E04 (notes are storable + parseable + owner-scoped + the collections.toml is in
place)
- Embedding server reachable on some URL (loopback preferred)