build · oxidb v0.25.21 0 entries on disk
The /dev/oxide

A build log on shipping OxiDB — notes, post-mortems, and the occasional flame war about JSON parsing, pressed straight onto an embedded engine running inside this process.

posts/0009.md · 2026-05-04

running OxiDB inside this gunicorn worker

hero image for: running OxiDB inside this gunicorn worker
asset · bucket: blog-images · key: e6841e6309dbfa265ba89521.jpg

This whole blog is one process. Gunicorn worker, eight threads, Django on top, OxiDB linked in as a shared library and called via ctypes. There is no oxidb-server running next to it. There is no SQLite next to it. Every `find`, every `put_object`, every `count` is a function call inside the same address space.

The wrapping is small. `oxidb-embedded-ffi` is a Rust crate that exports five C symbols: `oxidb_open`, `oxidb_open_encrypted`, `oxidb_close`, `oxidb_execute`, `oxidb_free_string`. The Python package (`oxidb-embedded`, on PyPI) is a single file of ctypes that binds those symbols and ships the `.dylib`/`.so`/`.dll` inside the wheel.

The wire format on the FFI side is the same length-prefixed JSON envelope the TCP server speaks — `{"cmd": "insert", "collection": "posts", "doc": {...}}` in, `{"ok": true, "data": {"id": 7}}` out. So the embedded client and the TCP client share their dispatch tables; new commands ship to both at once.

What it gives you:

* Zero IPC. No socket setup per request.

* One process to crash-recover, one log to tail.

* Deploys with the app — `tar czf` your data dir, scp it.

What it costs:

* One process per data dir, no exceptions. The B-tree, the WAL, the doc cache, and the FTS workers are all in-process state. Two processes opening the same dir race on writes and corrupt the tree.

For Django that translates to one rule: `gunicorn --workers 1 --threads N`. The Rust engine is internally thread-safe — per-collection RwLocks, scc::HashMap for lock-free bucket-level concurrency — so threads inside the one process are the right concurrency knob. The `bin/start.sh` in this repo runs `gunicorn --workers 1 --threads 8` and gets comfortable four-digit concurrent reads/sec on a laptop.

If you actually need multiple OS processes serving an OxiDB, boot `oxidb-server` and use the pure-Python `oxidb` TCP client. Different example, same API surface.