The data archiving and compression foundation of the COOLJAPAN ecosystem just got faster, broader, and leaner.
Today we released OxiArc 0.2.8 — hardware-accelerated checksums, a new disc-image format to read, and append that copies bytes instead of recompressing them.
No C. No Fortran. No zlib. No libarchive. No external shared libraries.
No FFI overhead. No build hell.
Just clean, memory-safe, blazing-fast archiving and compression that compiles to a single static binary and runs everywhere, including WASM.
Why 0.2.8 matters: speed, reach, and discipline
Three headline changes define this release.
Hardware-accelerated checksums. CRC is on the critical path of nearly every archive format, so OxiArc now reaches for the silicon. On Apple Silicon and other aarch64 chips, oxiarc-core computes SIMD CRC32 using PMULL instructions — and the result is bitwise-identical to the scalar path. On x86_64, oxiarc-snappy gains Snappy CRC32C via SSE4.2 (_mm_crc32_u64), with runtime feature dispatch through a OnceLock. No flags, no configuration — the fast path is chosen automatically.
ISO 9660 reading. OxiArc can now read .iso disc images. A new IsoReader parses the Primary Volume Descriptor and decodes Joliet UCS-2 filenames, with format detection via magic bytes at LBA 16. ISO 9660 joins the read-only family alongside 7z and CAB — it’s a read format, not a new write target.
Zero-copy, raw-preserve append. The oxiarc add command introduced in 0.2.7 now preserves ZIP and LZH entries byte-for-byte when appending, eliminating the decompress→recompress round-trip entirely. New low-level building blocks make this possible: ZipWriter::add_file_raw, LzhReader::read_raw_method_data, and LzhWriter::add_file_raw.
OxiArc remains API-stable as of v0.2.8: ~58,356 lines of pure Rust across 12 crates, with 1,281 tests passing.
What’s new
- SIMD CRC32 (aarch64 PMULL) in
oxiarc-core— hardware-accelerated, bit-identical to scalar. - Snappy CRC32C (SSE4.2) in
oxiarc-snappy— hardware-accelerated CRC32C for x86_64 via_mm_crc32_u64, dispatched at runtime withOnceLock. - ISO 9660 read support — new
IsoReaderwith PVD parsing and Joliet UCS-2 filenames;list,extract,info, anddetectall handle.isoimages. - Raw-preserve append — ZIP and LZH entries copied byte-for-byte in
oxiarc add(no recompress); addsZipWriter::add_file_raw,LzhReader::read_raw_method_data,LzhWriter::add_file_raw. - Progress and cancellation builders for the streaming codecs —
with_progress(Arc<dyn ProgressSink>)andwith_cancel(CancellationToken)onoxiarc-lz4,oxiarc-zstd, andoxiarc-lzmaencoders/decoders (e.g.Lz4Compressor,ZstdEncoder/ZstdStreamEncoder/ZstdStreamDecoder,Lzma2Encoder/Lzma2Decoder/Lzma2ChunkedEncoder). --memory-limit <BYTES>forextractandlist(accepts suffixes like100M,1G) to cap peak allocation per entry.
As always: zero clippy warnings (-D warnings), zero rustdoc warnings, and full COOLJAPAN policy compliance.
Technical Deep Dive: bit-identical SIMD and lossless append
The CRC work is the kind of optimization that has to be invisible to be useful. The carry-less multiply approach used by the aarch64 PMULL path folds the input in parallel, but it computes the exact same polynomial remainder as the scalar table-driven code — so existing archives validate identically and there is no “fast mode” caveat. The Snappy CRC32C path on x86_64 takes the analogous route through the dedicated _mm_crc32_u64 instruction, gated behind a OnceLock that probes the CPU once and caches the decision.
Raw-preserve append is the other half of the story. When you oxiarc add into a ZIP or LZH archive, OxiArc no longer needs to understand the content of the existing entries — only their boundaries. LzhReader::read_raw_method_data hands back the already-compressed method data, and ZipWriter::add_file_raw / LzhWriter::add_file_raw splice it into the output untouched. The payoff is that appending to an archive full of large, already-compressed entries is fast and lossless: no CPU spent recompressing data that was already compressed.
IsoReader rounds out the read-only container family in oxiarc-archive, sitting next to the 7z and CAB readers and wiring into the same CLI verbs.
Getting Started
Install the CLI:
cargo install oxiarc-cli
Inspect a disc image, cap memory on an untrusted archive, and append losslessly:
oxiarc list disk.iso
oxiarc extract data.tar.gz -o output/ --memory-limit 100M
oxiarc add archive.zip newfile.txt
For the library, add the archive crate plus the codec you need:
cargo add oxiarc-archive
cargo add oxiarc-deflate
use oxiarc_deflate::{deflate, inflate};
use oxiarc_archive::ZipReader;
use std::fs::File;
let compressed = deflate(b"Hello, World!", 6)?;
let decompressed = inflate(&compressed)?;
let file = File::open("archive.zip")?;
let mut zip = ZipReader::new(file)?;
for entry in zip.entries() {
println!("{}: {} bytes", entry.name, entry.size);
}
What’s New in 0.2.8
- SIMD CRC32 on aarch64 (PMULL) and Snappy CRC32C on x86_64 (SSE4.2) — automatic, bit-identical to scalar.
- ISO 9660 read support via
IsoReader(PVD + Joliet UCS-2);list/extract/info/detectaccept.iso. - Byte-for-byte raw-preserve append for ZIP and LZH in
oxiarc add— no decompress/recompress round-trip. with_progress/with_cancelbuilders on LZ4, Zstd, and LZMA encoders and decoders.--memory-limit <BYTES>(e.g.100M,1G) onextractandlistto cap per-entry allocation.- Zero clippy and rustdoc warnings; full policy compliance.
Tips
- Cap memory on untrusted input. Run
oxiarc extract --memory-limit 100M data.zip -o out/to bound peak allocation per entry and stay safe against decompression bombs. - Read disc images directly.
oxiarc list disk.iso(andextract/info/detect) work on.isofiles with Joliet long filenames decoded for you. - Append fast and lossless. Raw-preserve makes
oxiarc addcopy ZIP and LZH entries byte-for-byte — no recompression, ideal for archives full of already-compressed data. - Wire up progress and cancellation on long LZ4/Zstd/LZMA jobs with
with_progress(...)andwith_cancel(...)on their encoders and decoders. - SIMD is automatic. Hardware CRC kicks in on aarch64 (PMULL) and x86_64 (SSE4.2 CRC32C) with no flag, and the results are bit-identical to the scalar path — nothing to verify, nothing to opt into.
This is the foundation
OxiArc is the sovereign archiving and compression backend for the wider COOLJAPAN stack, and faster checksums plus lossless append directly benefit the pipelines that depend on it:
- OxiMedia — video/image asset packaging and distribution
- SciRS2 / NumRS2 — dataset compression and long-term storage
- OxiGDAL — geospatial file archiving
- ToRSh / OxiRAG — high-throughput data ingestion pipelines
- RusMES — mail attachment handling
Repository: https://github.com/cool-japan/oxiarc
Star the repo if you want high-performance archiving and compression — now with hardware-accelerated checksums and zero-copy append — and none of the traditional native toolchain headaches.
The era of “just use zlib” or “link libarchive” is coming to an end.
— KitaSan at COOLJAPAN OÜ May 8, 2026