posts/0015.md · 2026-04-28
GELF log ingestion — auto-index, retention, alerts
OxiDB doubles as a log destination. Set `OXIDB_GELF_PORT=12201` and it speaks GELF UDP. Graylog clients, Docker's `gelf` log driver, fluent-bit's gelf plugin — all just point and stream. Auto-decompresses chunked + gzipped GELF.
**Auto-indexing.** Incoming records get scanned for keys across the first N messages (`OXIDB_GELF_INDEX_FIRST`). Any field that appears in ≥ 50% of those messages gets a single-field index. New keys discovered later trigger an incremental index build in the background — no rebuild, no rescan. So queries like `find _gelf_logs where level=4 and host=...` are O(log n) from the moment they're useful.
**Retention.** TTL index on `_gelf_logs.timestamp` with `expireAfterSeconds=$OXIDB_GELF_RETENTION` does the delete-old-stuff job. Background scanner walks the index by date, deletes from the head. No cron, no separate pruner.
**Alerting.** `OXIDB_ALERT_INTERVAL=15` runs the alert evaluator every 15 s. Rules live in `_alerts`. Each rule is a query + a threshold + a webhook URL. The evaluator uses the same indexes the log queries do, so a rule like `count(_gelf_logs, level <= 3, last 60s) > 50` is a single index range scan.
**Companion CLI.** `oxidb-tail` (workspace crate) is a ratatui-based TUI that subscribes to the GELF collection's change stream and renders a colorized live tail. Looks like `lnav` but talks to OxiDB directly.