Nihilai Collective Logo

Jsonifier

Nihilai Collective

RFC8259-compliant JSON · SIMD + compile-time hash-maps · C++23

What It Is

Jsonifier is a set of classes for validating, serializing, parsing, prettifying, and minifying objects into and out of JSON strings — very rapidly. It is fully RFC8259 compliant, and is positioned as possibly the fastest JSON parser and serializer in C++.

It achieves this through the combined use of SIMD instructions and compile-time hash-maps for the keys of the data being parsed — so key lookup stays O(1) and order-independent rather than degrading on out-of-sequence documents. The structural index that drives every operation — parsing, serializing, minifying, prettifying — is stored as a compact array of buffer offsets rather than raw pointers, keeping the working set small and cache-friendly. Registered on the Microsoft vcpkg registry.

Why

Most fast JSON libraries are fast on the happy path and fragile everywhere else. Jsonifier is built to be fast and unbreakable. The serializer leans on compile-time reflection over your types, the parser resolves keys through a perfect hash computed at compile time, and the SIMD layer dispatches to the widest instruction set your CPU supports.

Jsonifier doesn't just parse JSON — it survives it. Every build is hardened against malformed input: every corpus document is truncated byte by byte down to empty and each truncation must fail cleanly, and the full jsonchecker and JSONTestSuite conformance corpora are run in every parse configuration. Zero crashes, zero memory violations, zero undefined behavior — under both AddressSanitizer and UndefinedBehaviorSanitizer, across the full Windows / Linux / macOS matrix, on every push.

Parsing, validation, minification, and prettification all fail fast now — the first invalid byte halts the operation immediately rather than letting the parser wander further into a broken buffer accumulating garbage state. Full UTF-8 validation runs as part of every parse by default, fused directly into string scanning rather than requiring a separate pass over the buffer.

Features

  • Fully RFC8259-compliant parsing and serialization
  • Parse, serialize, validate, prettify, and minify — one library
  • SIMD dispatch across AVX-512 / AVX2 / AVX / SSE / NEON / SVE2
  • Auto-detected CPU arch — x64, AVX, AVX2, AVX-512, PCLMULQDQ, ARM-NEON, ARM-SVE2 (experimental)
  • Runtime x64 tier dispatch — AVX, AVX2, and AVX-512 compiled into one binary, best tier picked via cpuid / xgetbv on first use
  • Cross-compilation by pre-defining JSONIFIER_CPU_INSTRUCTIONS (plus JSONIFIER_SVE2_VECTOR_BITS for SVE2) in CMake to skip native detection
  • Compile-time perfect hash-maps for key resolution
  • Hash-map strategy picked per object at compile time — dedicated 1-, 2-, and 3-key maps, plus single-byte, first-byte, and unique-byte-and-length keyed maps
  • Order-independent parsing — no penalty for out-of-sequence keys
  • Known-order mode: declaration-order key check, then a self-tuning per-position memo, then the hash map
  • Partial reading: extract only the keys you ask for
  • Schema-free, On Demand-style generic parser with re-readable values, out-of-order access, JSON Pointers, and iterateMany for NDJSON streams
  • Compile-time reflection — no macros, no schema boilerplate
  • Custom JSON key names (kebab-case, digit-prefixed, reserved words) via makeJsonEntity, resolved at compile time
  • Zero-copy parsing where possible
  • UTF-8 validation on by default, fused into string scanning
  • ASAN / UBSAN clean against a byte-level fuzz gauntlet
  • Custom parsing/serializing behavior per type
  • Parse arbitrary / unknown data via raw_json_data
  • Runtime key exclusion
  • Detailed error reporting with source-location tracking
  • Configurable max nesting depth (1024 by default), guarding against malicious input
  • Optimization mode for already-minified JSON
  • Cross-platform CI: Windows MSVC, Ubuntu GCC/Clang, macOS Clang/GCC

Quick Start

Define a type and parse into it
#include <jsonifier> struct Person { std::string name; int32_t age; double height; bool active; }; template<> struct jsonifier::core<Person> { using value_type = Person; static constexpr auto parseValue = createValue< &value_type::name, &value_type::age, &value_type::height, &value_type::active>(); }; int main() { jsonifier::jsonifier_core<> parser{}; std::string json = R"({"name":"John","age":30,"height":1.85,"active":true})"; Person person{}; parser.parseJson(person, json); }
Serialize back out
std::string out{}; parser.serializeJson(person, out); // out == {"name":"John","age":30,"height":1.85,"active":true}
Validate, prettify, minify
bool ok = parser.validateJson(json); std::string pretty{}; parser.prettifyJson(json, pretty); std::string minified{}; parser.minifyJson(pretty, minified);

Every error surfaces through the parser's error vector — see the Error Handling docs for the full reporting API. The jsonifier_core type is templated on an initial scratch-buffer size in bytes (1MB by default) for workloads dealing with unusually large documents. On x64 it holds one parser core per compiled SIMD tier and forwards each call to the best tier the running CPU supports, so a binary configured for AVX-512 also runs on AVX2 and AVX machines. Include only <jsonifier> — internal headers are not meant to be included directly.

Building

Consume via CMake's FetchContent, or install through vcpkg.

vcpkg

vcpkg install jsonifier

CMake (FetchContent)

include(FetchContent) FetchContent_Declare( Jsonifier GIT_REPOSITORY https://github.com/nihilai-collective/Jsonifier.git GIT_TAG main ) FetchContent_MakeAvailable(Jsonifier) target_link_libraries(your_target PRIVATE Jsonifier::Jsonifier)

Requirements

  • A C++23 compiler — MSVC 2022 v19.40+, GCC 14+, Clang 18+
  • CMake 3.28 or later
  • Supported CPU — x64, or ARM64 with NEON or SVE2 (SVE2 backend is experimental)

Testing & Safety

Every push triggers the full test suite across Windows MSVC, Ubuntu GCC, Ubuntu Clang, macOS Clang, and macOS GCC — with both AddressSanitizer and UndefinedBehaviorSanitizer live, treating warnings as errors under -Weverything / -Wpedantic / /Wall /W4.

Every parsing-family test runs in all eight partialRead × knownOrder × nullTerminated configurations, so every code path — full-document, partial, ordered, out-of-order, and null-terminated vs. length-delimited — gets the same coverage.

RFC8259 Conformance
265 fail-case + 122 pass-case documents (77 + 27 from the jsonchecker corpus, plus all 188 n_ + 95 y_ cases from the JSONTestSuite corpus), each checked against a specific expected error (the implementation-defined i_ cases are intentionally unused) and run in all eight partial × knownOrder × nullTerminated configurations — 3,096 assertions per platform.
Bounds & Truncation Fuzz
10 corpus documents (Canada, CitmCatalog, Discord, Google Maps, Instruments, Marine IK, Mesh, Random, Twitter, Twitter Partial), minified and prettified (20 total), each sliced byte-by-byte down to empty. Every truncation must fail parse cleanly. Run across all eight configs — 160 truncation runs.
Float Validation
64 edge cases: denormals, infinities, subnormal boundaries, extreme magnitudes, powers-of-two rounding cases, 46-digit mantissas. Run in all eight configurations.
Integer Bounds
Signed: 24 pass + 11 fail, covering INT64_MIN / INT64_MAX overflow. Unsigned: 16 pass + 11 fail through UINT64_MAX. All run in every configuration.
String & Unicode
35 pass + 26 fail cases: escape sequences, control characters, surrogate pairs, emoji, ZWJ sequences, unpaired surrogates, overlong escapes, truncated hex.
UTF-8 Validation
202 tests: standalone 1–4 byte sequence and chunk/block-boundary tests, the full Markus Kuhn UTF-8 stress-test corpus, second-byte boundary tests per lead-byte class, fused string-parser validation (escape-aware, cross-chunk, surrogate-pair-aware), an unaligned-pointer sweep, an unaligned invalid-sequence sweep, an mmap page-boundary fault check, and a width-transition sweep across body lengths 24–224.
SIMD Intrinsics
Direct correctness testing of the SIMD abstraction layer, independent of parsing: comparison, bitmask, logical, saturating-subtract, shift, cross-register byte-boundary alignment, and load/store round-trips at every unaligned offset, across every supported backend including the experimental SVE2 path.
Error Reporting & Core Lifecycle
10 tests for error construction, equality, line-number reporting, and control-character escaping in diagnostic output, plus 11 jsonifier_core copy / move / self-assignment tests checking that error lists and scratch buffers are copied, moved, and never aliased.
Round-Trip
27 serialize → parse → compare documents covering pointers, unique_ptr, nulls, empties, large numbers, exponent forms — 54 checks per configuration, run in all four partialRead × knownOrder configurations.
Reflection & Type Coverage
69 unit tests: basic reflection, renamed fields, optionals, enums as integers and map keys, tuples, shared_ptr, nested maps, vector-of-vectors, escaped keys, plus a dedicated partial-read suite (boundary lengths, large payloads, minified mode).
Parsing Corpus
Full real-world corpus (Twitter, Twitter Partial, CitmCatalog, Canada, Marine IK, Mesh, Discord, Google Maps, Instruments, Random), minified and prettified, parsed and re-serialized in every configuration, plus minify / prettify / validate passes.
Internal Containers & Utilities
624 tests for the library's own building blocks: the fixed-size allocator, jsonifier::array, the tuple implementation and its iterator, the compile-time hash and hash-map generators, comparators and the string-literal comparator, enum-name and member-name reflection, the jsonifier::string class, stage-1 tape emission, and the minifier / prettifier / printer output paths.
Generic Parsing
75 tests for the schema-free On Demand-style parser: in-order, reverse-order, and missing-key access, JSON pointers, escaped and non-ASCII strings, invalid-UTF-8 rejection, and multi-document streams.
raw_json_data
41 tests for raw_json_data: type detection for every JSON kind, int / uint / double round-trips, key and index access, contains, size, deep nesting, equality, and serialization (compact, prettified, and default-constructed).
Number Serialization
527 tests across the number writers: 76 fastio digit-count / integer / float tests, 83 zmij double-serialization tests, and 368 digit-boundary tests for i_to_str and jsonifier::toString across all eight integer types.
Sanitizer Coverage
ASAN with use-after-scope, sibling-call disabling, and _FORTIFY_SOURCE off. UBSAN with -fno-sanitize-recover=all so any UB fails the run. Static-linked libasan on Linux. Sanitizers are disabled for GCC on macOS (unsupported), and UBSan has no MSVC runtime.
Warnings-as-Errors
Clang builds under -Weverything -Wpedantic -Werror. GCC under -Wpedantic -Wextra -Wall -Werror. MSVC under /Wall /W4 /WX. Zero warnings tolerated.
Total Assertions
Over 7,000 individual assertions per platform per push (7,030 on Windows MSVC with the AVX2 backend), across five platform / compiler combinations, every one under both sanitizers.
Run the test suite locally
git clone https://github.com/nihilai-collective/Jsonifier.git cd Jsonifier cmake -B build -DJSONIFIER_UNIT_TESTS=ON cmake --build build --target jsonifier-unit-tests ./build/unit-tests/jsonifier-unit-tests

Add -DJSONIFIER_ASAN=ON or -DJSONIFIER_UBSAN=ON to enable sanitizers locally. Both are on by default in CI.

Documentation

InstallationFetchContent and vcpkg setup
Quick StartParsing and serializing real JSON in five minutes
ReflectionDescribing your types to the core template
Serializing / ParsingThe primary read and write APIs
Partial ReadingExtract only the fields you need
ValidatingRFC8259 conformance checking
Prettifying / MinifyingWhitespace-aware reformatting
Known OrderSelf-tuning fast path for documents with a stable key order
Optimizing for Minified JSONPerformance mode for already-minified input
Error HandlingInspecting parse and validation failures
UTF-8 ValidationCross-SIMD-width validation architecture
CPU Architecture SelectionRuntime x64 tier dispatch and manual instruction-set configuration, including SVE2 and cross-compilation overrides
Excluding Keys at RuntimeSkipping fields dynamically
Parsing Arbitrary Dataraw_json_data for unknown shapes
Generic ParsingSchema-free, On Demand-style parsing and NDJSON streams
Full usage guide on GitHub →

Benchmark Methodology

Throughput is measured in mebibytes processed per second (MiB/s) for both reading (parse) and writing (serialize), benchmarked against Glaze and simdjson on identical hardware, in the same binary. Each benchmark body reports the bytes it processed, and throughput is the total bytes over the total measured time of the scored window.

The harness uses adaptive epoch sampling. The exact iteration counts, RSE and convergence thresholds, and timing bounds for each run are posted alongside that run's results in the charts below. Each epoch runs a fresh batch of iterations, doubling every epoch, and is scored on a trailing window of k = clamp(iterations / 10, MinK, MaxK) samples. An epoch's score is its relative standard error (Bessel-corrected) plus its mean shift from the previous epoch, both in percent.

Sampling does not stop at the first good epoch. It keeps running until the time limit or the total iteration budget is exhausted, and the lowest-scoring epoch is kept as the canonical result, whether or not it met the convergence thresholds; the first epoch is kept only as an initial fallback. That result counts as converged only if its RSE and mean shift are both at or under their configured thresholds. Non-converged results are still included in the rankings and in win/tie/loss tallying, so any result that missed those thresholds should be read with its reduced statistical confidence in mind.

The Fastest / Statistical Ties / Runner-up summary counts Jsonifier's two variants (single-pass and two-stage) together as one best-of-variants entry: per test, only the better-ranked variant counts, and a test is a win when that variant places first with no other library sharing the position. simdjson's two variants (On Demand and reflection) are tallied the same way in the published results CSV. simdjson (ondemand) results include the harness's own field-mapping code that materializes values into the shared test structs.

All results are ranked by throughput. Neighbouring results are checked for a statistical tie with Welch's t-test on per-iteration timings, using Bessel-corrected variance, Welch–Satterthwaite degrees of freedom, and a two-sided 95% critical value (a small-sample-corrected approximation of the t distribution). Tied libraries share a rank, so a small delta inside the noise floor doesn't get dressed up as a victory.

CPU caches are evicted before each library's run, and the benchmark thread is pinned to a performance core (P-cores on hybrid Intel parts) with its priority raised (REALTIME / SCHED_FIFO / QOS_USER_INTERACTIVE). Each iteration processes a batch of pre-generated documents, per-iteration setup runs outside the timed region, and every output is consumed through do_not_optimize_away() so the compiler can't eliminate the work. Where the OS exposes them (perf_event on Linux, kperf on macOS), hardware counters are also recorded and reported per byte: cycles, instructions, branches, branch misses, cache references and cache misses.

Platforms: Windows, Linux, macOS — Compilers: MSVC, GCC, Clang — Harness: BenchmarkSuite

View the full benchmark repository →

Results

Benchmark Branch
Platform / Compiler
Loading benchmarks
Operation

Correctness Validation

Structural parity with simdjson, verified byte-for-byte across the entire benchmark corpus, plus dedicated bool, double, int64, string, and uint64 element tests. Each test document is parsed by both libraries; every emitted structural token is compared. A single mismatch fails the run.

Results are pulled from the server's correctness-validation data set for the selected test suite below.

Test Suite
Platform / Compiler
Loading correctness data

Engineering Papers

Deep-dive writeups on Jsonifier's architecture, written for readers who want the actual mechanism rather than the marketing summary.

Jsonifier Serialization: Design and Performance

Nihilai Collective Corp — Engineering Papers · October 2026 · Jsonifier

One size pass, one resize, then a write pass that never checks capacity. Key literals, separators and indentation are packed into integer constants at compile time, and nearly every copy has a compile-time power-of-2 size; bools are one subtraction and one store; integers are written 4 digits per store from compile-time tables; strings are escaped with SIMD, then SWAR. Fastest in 111 of 115 converged serialization tests against Glaze and simdjson's C++26 reflection writer across five platform/compiler builds.

Read the full paper →

Schema-Free, On Demand: Generic JSON Parsing in Jsonifier

Nihilai Collective Corp — Engineering Papers · September 2026 · Jsonifier

A schema-free, tape-driven parser shaped like simdjson's On Demand API, but re-accessible. In-order reads cost a cursor increment and a key compare; out-of-order reads go through a SIMD SwissTable-style field index that is only built once an object has skipped enough keys to earn it. Covers the shared forward cursor, zero-copy strings with a fixed-slot arena, why whole-document indexing made the target workload 2.3x slower, and the per-platform threshold sweep. Results across five platform/compiler builds: 289 wins, 15 ties, 44 losses against simdjson On Demand, including 69 wins and 2 losses in 76 full-document reverse-order tests, across in-order, reverse, sparse and sparse-reverse reads. A new section covers multi-document streams through iterateMany, the counterpart of simdjson's iterate_many: 84 wins, 4 ties, 9 losses across NDJSON and comma-separated streams, including 22 wins in 25 results on ports of simdjson's own streaming benchmarks and 28 wins in 32 reverse-order stream results.

Read the full paper →

Two Stages, On Demand: The Stage-1 + Stage-2 Architecture in Jsonifier

Nihilai Collective Corp — Engineering Papers · July 2026 · Jsonifier

Why Jsonifier treats the classic two-stage SIMD parsing model as a specialized tool rather than a mandatory front door — routing full-document parses through a single fused pass while reserving stage-1 structural indexing for partial reads, prettifying, and minifying. Covers the per-compiler step geometry, the fold-expression drain architecture for bitmask-to-index extraction, and the distributed in-register UTF-8 validation strategy, with direct comparisons to simdjson's current design throughout.

Read the full paper →

Batched Drain, Fused Scan: The Architecture of Jsonifier's Stage 1

Nihilai Collective Corp — Engineering Papers · July 2026 · Jsonifier

A point-by-point comparison of Jsonifier's stage-1 structural indexer against simdjson's: a batched drain that decouples mask production from tape emission, per-(ISA × compiler)-tuned step geometry, an AVX-512 compress-based extraction path, and UTF-8 validation fused directly into string unescaping and carried across SIMD width transitions via a compact three-byte validation state. Includes a case study on relocating control-character validation from stage 1 to stage 2, plus full cross-platform results — 99 wins, 1 tie, 12 losses against simdjson across 112 AVX2/NEON tests, with a perfect 23-0-0 sweep under MSVC, plus 56-1-12 across 69 tests on three AVX-512 targets.

Read the full paper →

Acknowledgments

Jsonifier stands on the shoulders of prior work in the high-performance JSON and numeric-conversion space. Credit where it's owed: