AkurAI Build
Sign in with AkurAI ID
Menu

popagent

public

Latest change 01ef89c245d0ecf6737050da659632f30e00d111 - Expose persisted Mastra traces by Ólafur Búi Ólafsson

# Observability

Mastra observability is self-hosted through `MastraStorageExporter` and a dedicated PostgreSQL v-next storage domain. `OBSERVABILITY_DATABASE_URL` is required and must never point at the application database.

## Signals and correlation

- Complete sampling records traces, automatic token/latency metrics, structured logs at info and above, and user feedback. Mastra currently persists spans without `serviceName`; read APIs therefore filter by bounded time/status only rather than hiding those rows behind a service-name predicate.
- Every chat turn, task, and schedule execution receives one valid Mastra trace
  ID. Chat metadata persists it under `metadata.custom.traceId` beside the stable
  turn ID. Task execution creates the trace before lifecycle start and carries
  that same ID through execution, completion, and pre-stream failure evidence.
- Only safe scalar request-context keys are extracted. Runtime settings and objects are excluded.
- The AI SDK finish event's provider usage is persisted under `metadata.custom.usage`; the UI reads token counts from that object.

## Privacy

Mastra's sensitive-data filter remains enabled. Secret-tool values have a persisted-span redaction contract, including generic `value` fields. Do not add raw prompts, credentials, browser profiles, hook payloads, runtime settings, or tool objects to trace metadata or logs.

## Read APIs and retention

Protected APIs expose overview KPIs, bounded trace lists/details, warning/error logs, health, and trace-linked feedback. Reads force a maximum 30-day window and bounded pagination. Daily maintenance retains ordinary signals for `POPAGENT_OBSERVABILITY_RETENTION_DAYS` (default 30) and feedback for `POPAGENT_OBSERVABILITY_FEEDBACK_RETENTION_DAYS` (default 90). Estimated cost is intentionally absent until 9Router model IDs are verified against Mastra's pricing registry.

The UI presents last-24-hour run/error/token/latency KPIs, expandable span timelines, correlated warnings/errors, explicit loading/empty/failure states, and trace-linked thumbs feedback.

## Contract ownership

- `src/observability.test.ts` — configuration, query bounds, correlation, redaction, and feedback contracts.
- `src/maintenance.test.ts` — signal and feedback retention.
- `src/server.test.ts` — protected health/read/feedback routes.
- `src/ui/ObservabilityPage.tsx` and final browser verification — routed UI behavior.

Observability storage failures must remain visible through health and structured stdout without redirecting writes to application PostgreSQL.