Two Stages, On Demand

Abstract

Since Langdale and Lemire's 2019 paper Parsing Gigabytes of JSON per Second, the two-stage SIMD parsing model — a vectorized structural-indexing pass (stage 1) followed by a tape-driven materialization pass (stage 2) — has been treated as the canonical architecture for high-performance JSON processing. simdjson, the reference implementation, routes every document through stage 1 unconditionally.

Jsonifier takes a different position: two-stage parsing is a specialized tool, not a mandatory front door. When parsing a full document into concrete, reflection-registered C++ types, every structural fact the tape would record is rediscovered anyway during value materialization — so building the tape means touching every byte twice for information used once. Jsonifier therefore parses full documents in a single fused pass, and reserves its stage-1 + stage-2 machinery for the workloads where a structural index genuinely pays for itself: partial/unordered reading, prettifying, and minifying.

This paper describes both halves of that architecture: the single-pass primary path, and Jsonifier's stage-1 implementation — including its per-compiler-tuned step geometry, its fold-expression drain architecture for bitmask-to-index extraction, and its distributed UTF-8 validation strategy — with direct comparisons to simdjson's current design throughout.

1. Background: the canonical two-stage model

simdjson's pipeline is well documented. Stage 1 sweeps the input in 64-byte blocks, computing per-block bitmasks: backslashes, odd-length escape terminations, quotes, an in-string range mask derived via carry-less multiplication (prefix XOR), structural characters ({ } [ ] : ,), whitespace, and pseudo-structural scalar starts. The surviving bits are converted to byte offsets — the structural tape — via a tzcnt/blsr extraction loop. Stage 2 then walks the tape to build a DOM or, in the On Demand API, lazily materializes only the values the caller touches.

The model's strength is that stage 1 is branch-free and data-parallel; its cost is that it is unconditional. Every byte of every document is classified and indexed before a single value is parsed, regardless of whether the caller's access pattern needs the index at all.

2. The Jsonifier position: pay for the tape only when the tape pays for you

Jsonifier's primary API parses JSON directly into user-defined structs registered through compile-time reflection (jsonifier::core<T>). In that setting the parser already knows, at compile time, the expected shape of the document: which keys exist, their serialized order, their types. The parse loop is a schema-directed walk, not a blind tree construction.

Consider what the structural tape provides: the positions of every brace, bracket, colon, comma, quote, and scalar start. Now consider what a schema-directed single-pass parser does at each of those same positions: it is already there, with its iterator parked on that exact byte, consuming it as part of matching a known key literal or delimiting a value. The tape's information is a strict subset of what the fused pass discovers for free in program order. Building it first means that, for a full-document parse in which every value is materialized:

• every byte of the input is loaded twice (once in stage 1, once in stage 2),
• the tape itself is written and re-read (cache traffic proportional to structural density), and
• stage 2's control flow is driven by indirect loads through the index array rather than by a pointer walking linearly through memory the prefetcher already understands.

(We scope the double-load claim deliberately: a two-stage consumer that skips content — simdjson's On Demand API skipping unrequested fields — does not reload the skipped bytes in its second pass. That is exactly the workload where the tape earns its keep, and exactly why Jsonifier retains the two-stage machinery for partial reading. The accounting above describes the full-materialization case, where nothing is skipped and the tape's information is fully redundant with the walk.)

For full-document parsing into known types, this is pure overhead. Jsonifier's measurements across its benchmark corpus consistently showed the fused single-pass path outperforming its own two-stage path on full-document workloads — which is precisely why the library ships both and routes between them.

The routing rule is simple: the two-stage machinery is engaged for partial reading, prettifying, and minifying — workloads where the caller does not want every value, or wants pure structural transformation. Full-document parsing takes the single-pass path.

One anticipated objection deserves preemption here: that Jsonifier's requirement of ahead-of-time registration (jsonifier::core<T>) concedes generality that simdjson retains, since simdjson parses arbitrary documents with no such declaration. For truly dynamic workloads — schemas unknown until runtime, exploratory traversal, structural transformation of unknown documents — this is correct, and simdjson's DOM and On Demand models are the appropriate tools; Jsonifier's registration model simply does not address that problem. But for the workload this paper concerns — parsing documents into concrete types the caller has defined — the objection dissolves on inspection, because the schema knowledge exists at compile time in both programs. A simdjson caller materializing a struct writes the schema into their source as a sequence of field accesses in a fixed order chosen at authoring time; that traversal code is a schema declaration in imperative clothing. The difference is not the presence of compile-time knowledge but its legibility to the library: expressed as hand-written traversal, the knowledge is opaque — simdjson cannot fuse key literals from it, cannot learn permuted orders through it, and cannot skip building the index it implies is unnecessary. Expressed as a reflection registration, the identical knowledge becomes architecture: fused member headers, adaptive order recovery, and the routing rule above. The comparison between the two libraries on known-type workloads is therefore not "declared schema versus no schema" — it is the same schema, declared once where the compiler can consume it versus restated per call site where it cannot.

A note on benchmarking, to keep this paper consistent with its stage-1 companion: the head-to-head results reported there are measured with Jsonifier's two-stage path explicitly engaged via the harness's branch-switching configuration, so that both libraries execute the same stage-1-then-stage-2 architecture and the comparison isolates the stage implementations themselves rather than the routing decision. As of the July 20, 2026 sweep, that record stands at 132 wins, 11 ties, 17 losses across 160 head-to-head tests against simdjson, including perfect 32-0-0 sweeps on two of the five platform/compiler targets:

The routing rule above describes the library's default behavior in production use; the benchmark deliberately overrides it for architectural comparability. Jsonifier's fused single-pass path is faster still on full-document workloads — which is the point of this section — and the harness's isolated-stage mode exists precisely to keep those two claims separately measurable. Per-platform tables and raw CSVs are maintained live at nihilai-collective.net; the figures above are a snapshot at time of writing.

3. The single-pass path: json_iterator over raw text

The fused path is implemented as json_iterator<parseOpts, string_view_ptr, string_buffer_type> — an iterator holding a raw const char* cursor plus depth counters, walking the document exactly once. Several design decisions distinguish it:

Compile-time key fusion. For each reflected member, the expected token is baked into the binary at compile time as a fused literal. In minified known-order mode, an entire member header — leading comma, quoted key, and colon — collapses into a single constant:

When the document arrives in the expected order — the overwhelmingly common case for machine-generated JSON — matching ,"name": is one constant-length comparison, and the cursor jumps the entire header in a single add. No tokenizer, no tape, no per-character state machine.

Adaptive out-of-order recovery. When the known-order guess misses, the parser falls back to a compile-time hash map over the type's keys (hash_map<value_type>::findIndex), dispatches through a generated table, and — crucially — learns: a thread-local antiHashStatesNew array records which index actually matched at each slot, so a document stream with a consistently permuted key order pays the hash cost once and then rides the corrected fast path.

This is worth dwelling on, because key order is where iterative traversal models pay their hidden tax. simdjson's On Demand API resolves a field lookup by scanning forward through the object from the current cursor position; the project's own documentation (doc/basics.md, field-access guidance) instructs callers to request fields in the order they appear in the document for best performance. When a requested key is not next, the cursor scans — and skips — every intervening key/value pair to find it, and a subsequent request for a key that lies behind the cursor forces a wrap-around rescan of the object. The degenerate case is a caller requesting fields in an order that consistently mismatches the document: each lookup can traverse a large fraction of the object's raw bytes, turning an O(members) parse into an O(members²) crawl over structural positions — per object, per document, forever. The model has no memory; the millionth permuted document costs exactly what the first one did.

Jsonifier's failure mode for the same situation is: one hash lookup, one dispatch-table indirection, and a learned correction. The mismatch cost is paid once per key slot per thread, not once per object instance. A feed of a billion documents with keys in reversed order parses at effectively the same throughput as a feed in declared order, because after the first document the "expected order" is the observed order. The schema-directed model converts key order from a per-document runtime tax into a per-stream calibration.

Whitespace memoization. The non-minified specialization exploits the fact that pretty-printed JSON repeats its indentation pattern on nearly every line. skipWhitespace records the first whitespace run it sees (wsStart, wsLength) and thereafter skips subsequent runs by comparing 8 bytes at a time against the memoized pattern via skipMatchingWs — turning per-character whitespace scanning into a couple of uint64_t XORs per line.

The result is a parser whose inner loop is dominated by wide constant comparisons and direct value materialization, with SIMD engaged surgically where it wins (string unescaping, discussed in §6) rather than as a mandatory preprocessing pass.

4. Stage 1: structural indexing, Jsonifier style

When the workload does justify a structural index, Jsonifier's stage 1 (simd_string_reader) implements the Langdale–Lemire bitmask algebra with several architectural departures.

4.1 Step geometry as a per-compiler compile-time constant

simdjson classifies input per 64-byte block, and its generic indexer steps through the buffer 128 bytes per iteration — two 64-byte blocks, software-pipelined one block deep, a geometry it retains even on its widest kernel (the Icelake implementation invokes index<128>). Jsonifier generalizes the unit of work to a step of simdBlocksPerStep 64-byte blocks, with the constant chosen per ISA and per compiler:

These values are not guesses. They were selected by Cartesian parameter sweeps across five platform/compiler CI targets (Windows/MSVC, Linux/GCC, Linux/Clang, macOS/Clang, plus ARM), validated against popcount histograms of structural density on the benchmark corpus. The finding that Clang and GCC want different geometry on the identical ISA — Clang preferring narrower steps with wider tape strides, GCC the reverse — reflects real differences in how each compiler schedules the unrolled block bodies, and is only expressible because the entire pipeline is specialized at compile time. A runtime-dispatched kernel gets one shape per ISA; a Cathedral-Architecture kernel gets one shape per (ISA × compiler) cell.

Processing up to 8 blocks (512 bytes) per step before draining amortizes the loop-carried state updates and gives the out-of-order core a deep window of independent block computations — a 2-4x wider scan window than simdjson's fixed 128-byte step.

4.2 The classification core: rope_detector and the collectors

The per-block bitmask algebra lives in small, composable functor structs. cmp_eq_op fans a broadcast comparison across the registers of a block and fuses the per-register movemasks into one uint64_t with compile-time shift amounts. The escape/quote/in-string state machine is rope_detector, a CRTP mixin over a plain rope_block of three masks:

The escape logic is the classic odd-length-backslash-run computation with the standard fast exit — a block containing zero backslashes skips the arithmetic and just consumes the carried nextIsEscaped bit, the same short-circuit simdjson's json_escape_scanner ships by default. Jsonifier extends the principle one level up: the prefix-XOR (clmul on x86, a shift-XOR ladder on NEON) that turns the quote mask into an in-string range mask is itself conditional — next branches on the quote mask, and a quoteless block bypasses the multiply entirely, inheriting prevInString directly via finishNextNoInString. Since long stretches of numeric or minified structural data contain no quotes at all, entire regions of such documents never touch the carry-less multiplier.

Whitespace and operator classification use the same shuffle-against-lookup-table trick simdjson pioneered, expressed as ws_collector and op_collector over the block's register array. On NEON, where movemask doesn't exist, the collectors use the vshrn_n_u16-based 4-bit-per-lane narrowing (with tzcnt >> 2 index correction in postCmpTzcnt) and, in the tuned NEON op_collector, a vqtbl1q_u8 nibble-shuffle keyed on (byte + 3) >> 4 — the reverse-bits/RBIT strategy is available as a compile-time switch where it profiles faster.

4.3 Pseudo-structural promotion

Scalar starts are promoted to structurals exactly as in the original algorithm — followsNonquoteScalar carries the cross-block bit — so the tape marks the first byte of every number, true/false/null, and string, giving stage 2 direct seek points to every value:

Note the overload pair: the minified variant omits the whitespace collector entirely — an entire classification lane deleted at compile time when the caller declares the input minified. This is the same philosophy as the compiler-specific step constants: every fact known before runtime is burned into the instruction stream.

4.4 The drain architecture: bits → indices

Extraction — converting each 64-bit structural mask into byte offsets on the tape — is where naive implementations serialize hard, because the classic loop (tzcnt, store, blsr, repeat) is a loop-carried dependency chain of length popcount.

Jsonifier's answer differs by ISA.

AVX-512: fully vectorized drain. With VBMI2 available, the bitmask never enters a scalar loop at all. _mm512_maskz_compress_epi8 compresses a constant 0..63 byte ramp under the structural mask, producing the set-bit positions as packed bytes; four cvtepu8_epi32 widenings plus a broadcast base-add stream up to 64 indices to the tape in a handful of instructions:

The widening cascade is guarded by the lane's precomputed popcount (count > 16, > 32, > 48), so sparse blocks pay for one store, not four. simdjson employs the same family of trick: its generic indexer exposes a CUSTOM_BIT_INDEXER hook, and the Icelake kernel fills it with its own VBMI2 compress-based extractor. On this ISA the two libraries again share the inner mechanism, and the performance separation comes from the surrounding step architecture (§4.1, §4.4 drain scheduling below) rather than the extraction primitive.

AVX2/AVX/NEON: the folded stepped drain. Without byte-compress, extraction must use the tzcnt chain — but the chain's structure is still a compile-time decision. write_indices_functor emits one extract-advance pair per index; write_indices_stepped_functor groups them into unconditional bursts of simdTapeStep writes:

The functor_runner's implAnd expands a stepped range sequence <0, 64, simdTapeStep> through an &&-fold: each group writes simdTapeStep indices unconditionally (over-writing garbage past the true count is harmless — the tape cursor only advances by the real popcount), and the fold short-circuits the moment a group's start index reaches the count. One predictable branch per simdTapeStep extractions instead of one per extraction.

Credit where due: simdjson's current generic bit_indexer implements the same stepped-burst pattern — write_indexes_stepped<START, END, STEP> via recursive template expansion, unconditional bursts, simdjson_unlikely short-circuit checks at each group boundary — with the burst size exposed as a build macro (SIMDJSON_STRUCTURAL_INDEXER_STEP, default 4). The two libraries have converged on the extraction inner loop itself. The divergences are in everything around it:

• Coverage. simdjson's stepped expansion runs to a fixed STEP_UNTIL of 24 set bits and falls back to a plain scalar loop for denser blocks; Jsonifier's fold covers the full 0..64 range at the same stride.
• Tuning axis. simdjson's STEP is one global knob; Jsonifier's simdTapeStep is swept and pinned per (ISA × compiler) cell, jointly with simdBlocksPerStep — the Clang-vs-GCC AVX2 split in §4.1 (stride 4 vs stride 1) exists precisely because the two constants interact.
• Drain scheduling. This is the structural difference. simdjson pipelines at a depth of one block: each next() call drains the previous block's structurals while the current block is being classified. Jsonifier defers draining for an entire step — up to eight masks and their popcounts are materialized in bitsArr/cntsArr before add_tape_values drains them back-to-back, each lane's tape destination precomputed from the popcount prefix. Eight independent tzcnt chains with no interleaved classification dependencies, handed to the out-of-order core as one batch.

Above the per-lane drain sits the same fold pattern at block scope. add_tape_values drains all blocks of a step, threading the running tape offset through a fold over the lane indices:

Because per-block popcounts were captured during classification (cntsArr[I]), the drains of successive blocks have no data dependence on each other's tzcnt chains — each lane knows its destination offset up front. The classification of blocks N+1..7 and the drain of block N are independent instruction streams the scheduler is free to interleave.

5. Stage 2: the tape-driven iterator

Stage 2 is not a separate parser — it is a specialization of the same json_iterator interface over structural_index_ptr instead of string_view_ptr. The reflection-driven parse machinery (parse_impl, the dispatch tables, the anti-hash learning) is written once against the iterator concept; a structural_context trait switches the token-navigation primitives:

This is the payoff that justifies the tape for partial reading: skipping an unwanted value is ++iter. Skipping an entire unwanted subtree touches only its structural characters — one indexed byte load per brace/bracket — never the bytes in between. skipString is a single increment, because stage 1 already resolved every escape sequence's effect on string extent. In the raw-pointer iterator, by contrast, skipping a string means re-scanning it for an unescaped closing quote (skipStringImpl's memchr-and-check-backslash-parity loop), and skipping a container means walking every byte.

The asymmetry defines the routing rule. When the caller wants every value (full-document parse into a reflected type), skips are rare and the tape is overhead. When the caller wants a few values from a large document (partial reading), or wants only the structure itself (prettify/minify, where the transformation is literally "copy bytes, adjusting whitespace at structural positions"), skips dominate and the tape converts O(bytes) navigation into O(structurals).

The same routing logic appears in stage 1's own entry point: reset<minified> selects between the whitespace-aware and whitespace-free classification pipelines, and the tail-block handling pads with 0x20, indexes the pad, then retroactively pops any tape entries pointing past the true document length — branchless main loop, exact tape.

6. Distributed UTF-8 validation

Let's be precise about what simdjson does, because it is also an in-register scheme: json_structural_indexer::next feeds every 64-byte block into utf8_checker::check_next_input using the very registers the classifier just loaded. Validation is fused into stage 1, costs no extra pass over memory, and covers the entire input. (The standalone generic_validate_utf8 exists too, for the buffer-validation API.) On Demand then additionally defers some string-level checks to traversal.

So the distinction is not "in-register versus dedicated pass" — both libraries validate in registers already in flight. The distinction is scope and location:

• simdjson validates all input bytes, in stage 1. Elegant and total, but it means the range checker runs over structural characters, whitespace, numbers, and literals — bytes that a JSON parser's own grammar already constrains to ASCII. For a document that is 70% non-string content, 70% of the validation work confirms what token matching would prove for free.
• Jsonifier validates string bytes only, in the string parse loop. Everything outside strings is implicitly ASCII-constrained: structural characters are matched literally, numbers pass through a digit-table parser, true/false/null are compared as packed integer constants — any non-ASCII byte in those positions is a parse error by construction, no range checker required. The only place arbitrary bytes can legally appear is inside strings, and that is exactly — and only — where utf8_register_validator runs.

One accounting qualifier belongs in the main text rather than a footnote, because it bounds the size of the claim: simdjson's checker leads with an ASCII fast path — a block whose registers OR to an ASCII-only result skips the multibyte carry chain entirely and pays roughly one reduction and one test. For ASCII-heavy input, then, "validation work proportional to input bytes" is proportional with a small constant, not with the full cost of the multibyte machinery. The scope distinction survives the qualifier — Jsonifier runs no validation instructions of any kind over non-string bytes, fast-path or otherwise, and the two approaches still separate cleanly on documents dense in non-ASCII string content — but the honest comparison states the fast path up front rather than burying it.

Jsonifier ships the same range-based algorithm simdjson uses (the byte1High/byte1Low/byte2High nibble-lookup classifier with the carry/tooShort/tooLong/surrogate error-bit algebra) in two deployments. The first, utf8_checker_new::validateUtf8, is the conventional standalone block validator, with the same ASCII fast path (orAll the block's registers, one test, skip the multibyte machinery). The second is the distributed one:

The validator carries prevInput/prevIncomplete across registers exactly as the block checker carries them across blocks, and checkPartial handles the sub-register tail at each quote or backslash boundary by padding with 0x41 (an innocuous ASCII byte) — so multi-byte sequences spanning register boundaries are still caught, and a string ending mid-sequence trips prevIncomplete.

The consequence differs by path. On simdjson's architecture, validation work is proportional to input bytes — every block, string or not, at minimum passes through the checker's ASCII test, and any block containing non-ASCII runs the full multibyte carry chain. On Jsonifier's, validation work is proportional to string bytes, and on the single-pass full-document path it rides registers the unescaper already loaded, so it never adds a memory pass and never touches non-string content at all. A document that is mostly numbers, structure, and whitespace validates nearly for free; the grammar itself is the validator for everything the range checker skips. There is also a timing difference: simdjson's stage-1 fusion produces its verdict at check_eof, after the whole input is scanned; Jsonifier's distributed scheme delivers a strict per-string verdict at each string's closing quote, so an invalid byte fails the parse at the value that contains it. It is enabled per-instantiation via parse_options::validateUtf8 — one more compile-time routing decision, selecting between two string_parser specializations, so callers who trust their input pay literally nothing.

7. Where the two libraries stand

The last row is the root of every other difference. simdjson must ship one binary that runs well everywhere, so its kernels are shaped for runtime selection among a fixed set. Jsonifier's founding constraint — only the data is a runtime variable; the schema, the ISA, the compiler, the parse options are all known at build time — lets every routing decision in this paper (tape or no tape, whitespace lane or not, validation or not, 4 blocks or 8, burst of 1 or 4) be resolved before the first byte is read.

8. Conclusion

The two-stage model is a genuinely great algorithm — Jsonifier's stage 1 is an unapologetic descendant of Langdale and Lemire's design, and credits it in source. The contribution here is architectural discipline about when to run it. A structural tape is an index, and indexes are worth building exactly when you will not read the whole book. Jsonifier builds it for partial reads and structural transforms, skips it for full parses, validates UTF-8 in the registers it was already holding, and lets the compiler specialize every remaining decision down to per-toolchain loop geometry. The benchmarks are the receipts.

"Whatever's already known when you write the code, let the compiler act on it." — Cathedral Architecture

References

Jsonifier is MIT-licensed and available at github.com/RealTimeChris/Jsonifier. Benchmark methodology and full sweep data: github.com/nihilai-collective/Json-Performance.