OxiArc 0.3.2 Released — Pure Rust AEC/SZIP for Scientific and Space Data

The codec that compresses satellite telemetry and HDF5 science datasets is now Pure Rust.

Today we released OxiArc 0.3.2 — a focused release built around one brand-new crate, oxiarc-szip: a complete Pure Rust implementation of AEC/SZIP (CCSDS-121.0-B-2).

No C. No Fortran. No zlib. No libarchive. No external shared libraries. No FFI overhead. No build hell. Just clean, memory-safe, blazing-fast archiving and compression that compiles to a single static binary, targets WASM, and runs everywhere.

OxiArc is the Pure Rust replacement for the zip, tar, gzip, zstd, and 7-zip tools — and for the Rust crates zip, flate2, zstd, bzip2, lz4, tar, snap, brotli, and miniz_oxide. With 0.3.2, that lineage extends into the scientific world.

Why 0.3.2 matters: AEC/SZIP, the science codec

AEC/SZIP is the lossless entropy codec underlying the HDF5/NetCDF SZIP filter (HDF5 filter ID 4). It is everywhere scientific and space data lives — CCSDS satellite telemetry, instrument products from NASA, ESA, and JAXA, FITS images, and NetCDF-4 datasets backed by HDF5. Until now, using it from Rust meant linking the C libaec reference library and dragging a native build into your toolchain.

oxiarc-szip is a from-scratch Pure Rust implementation of full AEC/SZIP encode and decode, compliant with CCSDS-121.0-B-2 and compatible with the libaec reference library. The API is deliberately small:

• encode compresses a &[u64] sample array into an AEC/SZIP bit stream.
• encode_bytes is a convenience wrapper for when you have raw &[u8].
• decode decompresses an AEC/SZIP byte stream back to raw sample bytes.
• SzipParams configures every knob; SzipError is the typed error enum.
• BitReader / BitWriter handle the bit-level manipulation underneath.

Round-trip tests cover a variety of sample scenarios across the parameter space.

A focused technical note on the parameters

The whole point of AEC/SZIP is matching the codec to the shape of your data, and SzipParams exposes all of it:

• bits_per_pixel (1–32; commonly 8 / 16 / 32) — the sample width.
• pixels_per_block (8 / 16 / 32) — the J parameter, the block size the codec adapts over.
• reference_sample_interval — how often a restart point is emitted.
• msb — MSB vs LSB bit ordering.
• nn_preprocess — the NN unit-delay predictor preprocessing step; the decoder applies the inverse automatically.
• rsi_byte_align — RSI byte alignment, for hardware AEC and the HDF5 AEC_CHIP option.

Because the implementation is faithful to CCSDS-121.0-B-2 and libaec, the output is byte-for-byte libaec-compatible — streams produced by OxiArc interoperate cleanly with existing HDF5 and NetCDF SZIP tooling, and vice versa.

This release also nudges the project’s footprint to roughly 72,000 SLoC total with 1,666 tests passing, all COOLJAPAN policies compliant.

Getting Started

cargo add oxiarc-szip

A complete encode/decode round-trip:

use oxiarc_szip::{SzipParams, decode, encode};

fn roundtrip() -> Result<(), oxiarc_szip::SzipError> {
    let params = SzipParams {
        bits_per_pixel: 8,
        pixels_per_block: 8,
        samples: 16,
        reference_sample_interval: 8,
        msb: true,
        nn_preprocess: false,
        rsi_byte_align: false,
    };

    let samples: Vec<u64> = (0..16u64).collect();
    let compressed = encode(&samples, &params)?;
    let raw_bytes  = decode(&compressed, &params)?;

    let decoded: Vec<u64> = raw_bytes.iter().map(|&b| b as u64).collect();
    assert_eq!(decoded, samples);
    Ok(())
}

What’s New in 0.3.2

• A brand-new oxiarc-szip crate: full Pure Rust AEC/SZIP encode and decode, compliant with CCSDS-121.0-B-2 and compatible with libaec.
• encode / encode_bytes / decode entry points, plus BitReader / BitWriter primitives and a typed SzipError.
• SzipParams exposing the full set of AEC/SZIP knobs: bits per pixel, pixels per block (J), reference sample interval, MSB/LSB ordering, NN unit-delay preprocessing, and RSI byte alignment.
• Byte-for-byte compatibility with libaec, so streams interoperate with HDF5/NetCDF SZIP tooling.
• Round-trip test coverage across varied sample scenarios.
• ~72,000 SLoC total, 1,666 tests passing, all policies compliant.

Tips

• Match bits_per_pixel to your sample width (8 / 16 / 32) — a mismatch corrupts the stream.
• Enable nn_preprocess for smoothly-varying telemetry or imagery to improve the ratio via the unit-delay NN predictor; the decoder inverts it for you automatically.
• Trade compression against restartability by tuning pixels_per_block (J = 8 / 16 / 32) together with reference_sample_interval.
• Set rsi_byte_align when targeting hardware AEC or the HDF5 AEC_CHIP option.
• Interoperate freely: output is byte-for-byte libaec-compatible, so streams flow both ways with existing HDF5 and NetCDF SZIP tooling.
• Reach for encode_bytes when you already hold a raw &[u8] rather than a &[u64] sample array.

This is the foundation

oxiarc-szip makes OxiArc a natural fit for the scientific side of COOLJAPAN. It speaks the same format as the HDF5/NetCDF SZIP filter, which makes it an obvious storage layer for SciRS2 / NumRS2 scientific datasets and HDF5/NetCDF-style archives — sitting alongside OxiMedia for asset packaging, OxiGDAL for geospatial files, ToRSh / OxiRAG for ingestion pipelines, and RusMES for attachments. When your data comes off a satellite or out of an instrument, OxiArc can now compress and decompress it the standards-compliant way, in Pure Rust, with no native toolchain to vendor.

Repository: https://github.com/cool-japan/oxiarc

Star the repo if you want standards-compliant scientific compression without the traditional native toolchain headaches.

The era of “just use zlib” or “link libaec” is coming to an end.

— KitaSan at COOLJAPAN OÜ June 1, 2026