OxiArc 0.4.0 Released — The Decoder Stops Writing Every Byte Twice

The fastest way to decode a byte is to write it exactly once.

Today we released OxiArc 0.4.0 — a DEFLATE/zlib decoder performance rewrite. No archive or stream wire format changed, no public API was removed, and no other codec crate in the workspace was touched: existing callers of inflate, Inflater::new, and zlib_decompress get the same bytes back, just faster.

No C, no Fortran, no zlib, no libarchive, no external shared libraries. Just clean, memory-safe archiving and compression that compiles to a single static binary, targets WASM, and runs everywhere. OxiArc is the Pure Rust replacement for the zip, tar, gzip, zstd, and 7-zip tools — and for the Rust crates zip, flate2, zstd, bzip2, lz4, tar, snap, brotli, and miniz_oxide.

Why OxiArc 0.4.0 is a game changer

The old inflate path was correct — 0.3.6’s 75-item interop campaign had already proven that byte-for-byte against the reference implementation — but it left real throughput on the table:

• It pulled bits from the stream one Read::read call at a time instead of prefetching in bulk.
• It wrote every decoded byte twice: once into a separate LZ77 ring buffer, once into the output.
• Its Huffman table was a single flat 9-bit table, not the two-level root+sub-table layout zlib and libdeflate use.
• Adler32::update processed input one byte at a time instead of in vectorizable groups.

OxiArc 0.4.0 ends all of that:

• A buffered BitReader. BitReader::buffered/with_buffer_capacity refill the bit accumulator via bulk 64-bit little-endian loads instead of per-bit reads from the underlying Read. BitReader::new (exact mode, never reads ahead) is unchanged and still required wherever the reader must not advance past bits actually consumed — e.g. ZIP’s byte-aligned data descriptor immediately following a DEFLATE member.
• A register-resident BitCache. BitReader::detach/reattach/refill_cache let a decoder’s inner loop hold the bit accumulator locally and decode many symbols against it, removing the store/load-forwarding stall a memory-resident accumulator costs on every single symbol.
• A two-level Huffman table. HuffmanTree now decodes through a root+sub-table layout — the root widened from a single-level 9-bit table to a 10-bit root — in the style of zlib’s inflate_table/libdeflate, instead of one flat table.
• The LZ77 history is the output buffer. InflateWindow, backed by Vec::extend_from_within, replaces the separate ring buffer that required writing every decoded byte twice.
• A vectorizable Adler-32. Adler32::update now folds 32-byte groups through a closed-form reduction (b' = b + 32·a + Σ(32-i)·xᵢ) instead of one add-pair per byte, letting the compiler auto-vectorize it — the same shape as zlib’s DO16 unrolling. Output stays bit-identical to the old byte-at-a-time version.

Technical Deep Dive: the inflate stack, layer by layer

1. Bulk bit supply. BitReader::buffered(with_buffer_capacity) prefetches via 64-bit LE loads; BitCache keeps the accumulator register-resident across a decode loop instead of round-tripping it through memory per symbol.
2. Two-level symbol decode. A 10-bit root Huffman table with sub-tables for longer codes, replacing the old flat 9-bit table — the same shape zlib and libdeflate converged on for a reason.
3. Single-write history. InflateWindow treats the growing output Vec itself as the LZ77 back-reference window (Vec::extend_from_within), so a match copy is one write, not two.
4. Zero-copy entry points. inflate_into(src, dst) and zlib::zlib_decompress_into decode straight into a caller-owned buffer — no intermediate Vec, no output-size guessing, and BufferTooSmall instead of silent truncation if the stream would overflow dst.

Every layer is covered by a new differential suite (oxiarc-deflate/tests/inflate_differential.rs) that proves the buffered fast path, the exact-mode path, and the _into APIs all agree byte-for-byte — across stored/fixed/dynamic blocks, maximum-distance (32 KiB) back-references, and hostile/truncated/corrupted input — plus an optional CPython zlib oracle comparison behind the pre-existing zlib-oracle feature, and a new fuzz_inflate_into target cross-checking the growable-Vec and slice-sink decode paths.

Getting Started

cargo add [email protected]

use oxiarc_deflate::{deflate, inflate_into};

let original = b"Hello, World! Hello, World!";
let compressed = deflate(original, 6)?;

// Decompress directly into a caller-supplied buffer -- no intermediate Vec,
// no output-size guessing. A stream that would overflow `out` returns
// BufferTooSmall instead of truncating silently.
let mut out = vec![0u8; original.len()];
let n = inflate_into(&compressed, &mut out)?;
assert_eq!(&out[..n], original);

zlib::zlib_decompress_into is the zlib-wrapped equivalent, additionally verifying the trailing Adler-32 checksum.

What’s New in 0.4.0

• inflate_into(src, dst) -> Result<usize> and zlib::zlib_decompress_into — zero-copy decode into a caller-supplied buffer.
• BitReader::buffered/with_buffer_capacity — buffered/prefetch reader mode with bulk 64-bit LE refills.
• BitCache, plus BitReader::detach/reattach/refill_cache — a register-resident bit accumulator for decoder inner loops.
• BitReader::into_parts/buffered_len — recover prefetched-but-unconsumed bytes so a buffered reader can hand a shared stream back to other code.
• Inflater::with_output_capacity(size_hint) and MAX_OUTPUT_CAPACITY_HINT (64 MiB clamp) — pre-size the decoder’s output buffer from an untrusted size hint; GZIP decoding now seeds this automatically from the trailing ISIZE field.
• HuffmanTree two-level root+sub-table decode (10-bit root); InflateWindow output-buffer-as-history; vectorizable Adler32::update.
• zlib_decompress internals split into zlib_payload (header validation) and verify_zlib_trailer (Adler-32 check), now shared with zlib_decompress_into.
• The Huffman fast-decode path’s last unsafe/get_unchecked table access is now safe, bounds-checked code.
• New fuzz target fuzz_inflate_into and new differential suite tests/inflate_differential.rs.
• 2,468 tests passing, 0 failed (2,329 via nextest across 101 binaries + 139 doctests, 13/13 crates green); zero clippy warnings (--all-features --all-targets -D warnings); zero rustdoc warnings; cargo fmt --all --check clean; cargo audit clean; cargo deny check bans clean; ~116,354 Rust lines across 339 files (tokei).

Only oxiarc-core and oxiarc-deflate changed source this cycle — the other 11 crates are unchanged from 0.3.6.

Tips

• Reach for inflate_into/zlib_decompress_into whenever you already own the output buffer. Reusing a Vec across many decode calls skips the allocation inflate/zlib_decompress do internally.
• Keep using BitReader::new (exact mode) anywhere byte-alignment matters right after a DEFLATE member — ZIP’s data descriptor is the canonical case. Reach for BitReader::buffered/with_buffer_capacity when the reader owns its stream and can read ahead freely.
• Size decode buffers with Inflater::with_output_capacity(size_hint) instead of guessing — it’s clamped to MAX_OUTPUT_CAPACITY_HINT (64 MiB) so a hostile size hint can’t force an unbounded pre-allocation. GZIP decoding already does this for you via the trailing ISIZE field.
• BufferTooSmall replaces silent truncation on inflate_into/zlib_decompress_into — if you were previously guessing output sizes, switch to checking for this error instead of assuming success meant “all of it decoded.”
• Nothing to migrate if you only call inflate, Inflater::new, or zlib_decompress. Same signatures, same output, just a faster decode loop underneath.
• If you have CPython available in CI, enable the zlib-oracle feature — the new inflate_differential.rs suite will cross-check every decode path against it for free.

This is the foundation

A faster, still byte-identical DEFLATE/zlib decoder benefits everything downstream that reads gzip, zlib streams, or ZIP archives without changing a single call site. NumRS2, SciRS2, ToRSh, RusMES, FVRS, TrustFormers, SkLearS, OxiGeo, and OxiMedia all pin oxiarc-* crates for archive and compression I/O — none of them need a code change to pick up this release’s throughput, only a version bump.

Repository: https://github.com/cool-japan/oxiarc

Star the repo if you want a decoder that got faster without asking you to touch a single call site.

The era of decoding every byte twice is over. Pure Rust archiving that’s fast, safe, and sovereign — is here.

— KitaSan at COOLJAPAN OÜ July 30, 2026