posts/0009.md · 2026-05-04
running OxiDB inside this gunicorn worker
This whole blog is one process. Gunicorn worker, eight threads, Django on top, OxiDB linked in as a shared library and called via ctypes. There is no oxidb-server running next to it. There is no SQLite next to it. Every `find`, every `put_object`, every `count` is a function call inside the same address space.
The wrapping is small. `oxidb-embedded-ffi` is a Rust crate that exports five C symbols: `oxidb_open`, `oxidb_open_encrypted`, `oxidb_close`, `oxidb_execute`, `oxidb_free_string`. The Python package (`oxidb-embedded`, on PyPI) is a single file of ctypes that binds those symbols and ships the `.dylib`/`.so`/`.dll` inside the wheel.
The wire format on the FFI side is the same length-prefixed JSON envelope the TCP server speaks — `{"cmd": "insert", "collection": "posts", "doc": {...}}` in, `{"ok": true, "data": {"id": 7}}` out. So the embedded client and the TCP client share their dispatch tables; new commands ship to both at once.
What it gives you:
* Zero IPC. No socket setup per request.
* One process to crash-recover, one log to tail.
* Deploys with the app — `tar czf` your data dir, scp it.
What it costs:
* One process per data dir, no exceptions. The B-tree, the WAL, the doc cache, and the FTS workers are all in-process state. Two processes opening the same dir race on writes and corrupt the tree.
For Django that translates to one rule: `gunicorn --workers 1 --threads N`. The Rust engine is internally thread-safe — per-collection RwLocks, scc::HashMap for lock-free bucket-level concurrency — so threads inside the one process are the right concurrency knob. The `bin/start.sh` in this repo runs `gunicorn --workers 1 --threads 8` and gets comfortable four-digit concurrent reads/sec on a laptop.
If you actually need multiple OS processes serving an OxiDB, boot `oxidb-server` and use the pure-Python `oxidb` TCP client. Different example, same API surface.