AkurAI-Framework / crates/llm/src/lib.rs

Latest change a81af83654d182e3becb8e2beaed6d61a3bd8e6b - feat(llm): streaming OpenAI-compatible client + /api/assistant SSE endpoint by Ólafur Búi Ólafsson

//! AkurAI-Framework llm — a pure-`std` streaming client for any
//! OpenAI-compatible `/v1/chat/completions` endpoint.
//!
//! Dependency-free (only the workspace's own `akurai-json`). It POSTs a chat
//! request with `"stream": true` and reads the **SSE** response of
//! `data: {chat.completion.chunk}` lines, yielding text deltas and assembled
//! tool calls through a callback:
//!
//! ```no_run
//! use akurai_llm::{stream_chat, ChatEvent, Message};
//!
//! let messages = [Message::new("user", "Hello!")];
//! stream_chat("http://localhost:8080", "gemma", &messages, None, |event| match event {
//!     ChatEvent::Delta(text) => print!("{text}"),
//!     ChatEvent::ToolCall { name, .. } => eprintln!("tool: {name}"),
//!     ChatEvent::Done => println!(),
//! })
//! .unwrap();
//! ```
//!
//! **No TLS.** std can't speak TLS and we will not pull a crate for it; TLS
//! terminates at the edge (Caddy/nginx) and this client points at the
//! plain-HTTP origin. An `https://` endpoint returns
//! [`LlmError::TlsUnsupported`].
//!
//! The crate is **config-free**: endpoint and model are always parameters. The
//! CLI reads `AKURAI_LLM_URL` / `AKURAI_LLM_MODEL` from the environment and
//! relays events as SSE on `/api/assistant`.
//!
//! The **pure** parts — endpoint parsing, request building, SSE chunk parsing,
//! and tool-call assembly — are factored out and unit-tested against
//! hand-written strings. The socket round-trip ([`stream::stream_lines`]) is
//! not unit-tested (tests never touch the network).

#![forbid(unsafe_code)]

mod client;
mod error;
mod parse;
mod request;
mod stream;
mod types;

pub use client::stream_chat;
pub use error::LlmError;
pub use parse::{parse_chunk, sse_data_payload};
pub use request::{build_body, build_request, parse_endpoint, Endpoint};
pub use types::{ChatEvent, Chunk, Message, ToolCallAccumulator, ToolCallDelta};