build · oxidb v0.25.21 0 entries on disk
The /dev/oxide

A build log on shipping OxiDB — notes, post-mortems, and the occasional flame war about JSON parsing, pressed straight onto an embedded engine running inside this process.

posts/0010.md · 2026-05-03

why JSONB and not JSON text

hero image for: why JSONB and not JSON text
asset · bucket: blog-images · key: db85d4ed767192b84e9879ea.jpg

OxiDB stores documents as JSONB — Postgres-style binary JSON — not as JSON text. That choice pays off in exactly one place and it pays a *lot*: partial extraction.

JSONB has a header section that's an offset table. Field lookups jump straight to the value's offset without parsing the rest of the document. JSON text can't do this — the parser has no way to know where field N lives without scanning past fields 0 through N-1.

Concretely, on a realistic 13 KB document with 50 nested events plus scalars and arrays:

serde_json::from_slice → Value (full decode): 27 µs

jsonb full decode → Value: 27 µs

jsonb partial extract → IndexValue: 90 ns

Three hundred times faster, doc-size-independent. That's what index build, sort-key extraction, group-key extraction, and `$eq`/`$gt`/`$exists` predicate evaluation all run on now. The hot path of the query engine never touches the Value tree.

The decode codec auto-detects: if `bytes[0]` is `{` (0x7B) or `[` (0x5B), it's JSON text and goes through `serde_json::from_slice`. Otherwise it's JSONB binary and goes through `jsonb::RawJsonb`. That lets us read legacy JSON-text `.dat` files written by older binaries without a migration step.

Custom deserializer? I tried it. Profiled a `from_raw_jsonb_to_index_value` that skips serde entirely. Came out **3–5% slower**, not faster. The reason: `get_by_keypath` is the dominant cost (~90 ns), and the scalar dispatch after it is so cheap that the serde Deserialize indirection doesn't even register. The lesson: measure before you write a custom anything. The bench code is in `examples/profile_decode.rs` in the repo, runs in five seconds, deletes opinions.