AkurAI-Framework / site/content/docs/sql.md

Latest change 5a1781679346dc042a413c85c2ddfc034f219ae0 - feat(sql): lexer + parser for a SQL subset (AST) by Ólafur Búi Ólafsson

# SQL

The framework is growing its own query engine — a from-scratch SQL
implementation in pure Rust, zero dependencies, like everything else here. This
page documents the first slice: the **lexer and parser**. They turn SQL text
into an abstract syntax tree (AST). Running that AST over the storage engine is
the next step (see [Roadmap](/docs/roadmap)).

## The front half of the engine

`crates/sql` is **parser-only** today. A query travels through two stages:

1. **Lexer** (`tokenize`) — splits the SQL text into a stream of position-tagged
   tokens: keywords (case-insensitive), identifiers, string and number literals,
   operators, and punctuation. Whitespace is skipped; every token remembers the
   character offset where it began, so errors can point at the exact spot.
2. **Parser** (`parse`) — a recursive-descent parser that consumes the tokens
   and builds a `Statement` AST node. Malformed input returns a positioned
   error, never a panic.

```rust
use akurai_sql::{parse, Statement};

let stmt = parse("SELECT id, name FROM users WHERE age >= 18 LIMIT 10").unwrap();
assert!(matches!(stmt, Statement::Select(_)));
```

The public API is a single function plus the AST types:

```rust
pub fn parse(sql: &str) -> Result<Statement, SqlError>;
```

## Supported subset

Three statement kinds are parsed today:

```sql
-- Query
SELECT id, name FROM users
WHERE age >= 18 AND (active = 1 OR role = 'admin')
ORDER BY name DESC
LIMIT 10;

-- Insert
INSERT INTO users (id, name) VALUES (1, 'Ann');

-- Create
CREATE TABLE users (id INTEGER NOT NULL, name TEXT, score REAL);
```

- **SELECT** — `*` or a comma-separated column list, a single `FROM` table, and
  optional `WHERE`, `ORDER BY <col> [ASC|DESC]` (defaults to `ASC`), and
  `LIMIT <n>` clauses.
- **INSERT** — an explicit column list and a matching `VALUES` list of literals.
- **CREATE TABLE** — column definitions of `<name> <type> [NOT NULL]`, where the
  type is `INTEGER`, `TEXT`, or `REAL`.

### WHERE expressions

`WHERE` is a real expression tree, not a flat list of conditions:

- **Comparisons:** `= != <> < <= > >=` between a column and a literal (in either
  order).
- **Boolean combinators:** `AND`, `OR`, and `NOT`, with parentheses for grouping.
- **Precedence:** `AND` binds tighter than `OR`. So `a=1 AND b=2 OR c=3` parses
  as `(a=1 AND b=2) OR c=3`. Parentheses override this.

Literals are `NULL`, integers, floats, and single-quoted strings. Inside a
string, two single quotes (`''`) are an escaped quote: `'it''s ok'` is the value
`it's ok`.

## The AST shape

`parse` returns a `Statement`:

```rust
pub enum Statement {
    Select(SelectStmt),
    Insert(InsertStmt),
    CreateTable(CreateTableStmt),
}
```

The key types underneath it:

- `SelectStmt { columns, table, where_clause, order_by, limit }`, where
  `columns` is a `Vec<SelectColumn>` (`All` for `*` or `Named(String)`),
  `where_clause` is an `Option<Expr>`, `order_by` is an `Option<OrderBy>`, and
  `limit` is an `Option<i64>`.
- `InsertStmt { table, columns: Vec<String>, values: Vec<Literal> }`.
- `CreateTableStmt { table, columns: Vec<ColumnDef> }`, where each `ColumnDef`
  is `{ name, ty: DataType, not_null: bool }`.
- `Expr` is the `WHERE` tree: `Literal`, `Column`, `Comparison { left, op, right }`,
  `And`, `Or`, and `Not`.
- `Literal` is `Null | Int(i64) | Float(f64) | Str(String)`.

## Errors

Any lex or parse failure is a `SqlError { message, position }` — a clear message
plus the 0-based character offset in the source. The parser is total: bad input
yields an `Err`, never a panic. Missing `FROM`, an unclosed paren, an unknown
statement keyword, or a stray character all surface as positioned errors.

## What's next

This crate stops at the AST. The next slice is the **executor**: walking the
parsed `Statement` against the [storage engine](/docs/storage) B+tree — a
catalog of tables and columns, evaluating `WHERE` over each row, applying
`ORDER BY` and `LIMIT`, and materializing `INSERT`/`CREATE TABLE`. The parser's
job is to hand that executor a clean, validated tree to work from.