◉ COOLJAPAN
← All posts
July 21, 2026 • 10 min read

OxiH5 0.2.1 Released — Compression, In-Place Overwrite, Nested Groups, and Fixing Every Chunked Dataset It Ever Wrote

OxiH5 0.2.1 adds DEFLATE compression on write, per-chunk tiling, in-place dataset overwrite, and nested groups — and fixes 9 correctness bugs, including a chunk B-tree defect that made every chunked dataset the writer produced unreadable by libhdf5. 682 tests passing — the sovereign scientific-data layer for COOLJAPAN.

release oxih5 pure-rust cooljapan noffi hdf5 scientific-data file-format io

A chunk B-tree node sized for the one chunk it held. That single defect meant every chunked dataset OxiH5’s writer had ever produced was unreadable by libhdf5 — and it’s just one of nine correctness bugs 0.2.1 fixes, alongside five major new write-path features.

Today we released OxiH5 0.2.1 — a release that adds DEFLATE/gzip compression on write, genuine per-chunk tiling for unlimited-dimension datasets, non-destructive in-place dataset overwrite, groups nested to any depth, and attributes on groups and arrays — while fixing nine correctness defects across the write and read paths, several of them severe enough to make previously-written files unreadable by the reference implementation.

No libhdf5. No FFI. No -sys crates. OxiH5 parses and now writes the real HDF5 binary format — superblock, object headers, B-trees, heaps, filter pipelines, all eleven datatype classes — using nothing but std. #![forbid(unsafe_code)] guards oxih5-core; the facade crate carries exactly one documented unsafe block, the mmap call behind open_mmap. It compiles to a single static binary, needs no system libraries and no C toolchain at build time, and stays WASM-friendly by construction.

Why OxiH5 0.2.1 is a game changer

0.2.0 hardened the reader against real-world superblocks. 0.2.1’s audit turned to the write path — and to library-wide assumptions about how many links a group can hold — and found defects that had been there since the writer’s first line:

OxiH5 0.2.1 ends all of that — and adds the write-side features those fixes unlocked:

Technical Deep Dive: how the write-path and layout-v4 fixes fit together

  1. Chunk B-tree node width — oxih5-format. Nodes are now emitted at the full width libhdf5 reads before it has seen their contents (2096 bytes for 1-D, 2616 for 2-D, 3136 for 3-D, plus a fixed increment per index level past 64 chunks), with unused slots zero-filled and the terminal key carrying the real dataset extent instead of being left at zero.
  2. The compression pipeline runs during layout, not emit. A compressed dataset’s byte length isn’t derivable from anything the caller supplied, and the chunk B-tree key has to record it before any byte is written — so a new write/payload.rs owns data_size() for both passes to agree on, and a new write/pipeline.rs encodes the filter-pipeline message (0x000B).
  3. Group symbol tables are chunked and B-tree indexed. Links are now split into 328-byte SNOD nodes — the width libhdf5’s default leaf_node_K = 4 actually implies — indexed by a group B-tree that grows a level once 32 nodes fill, verified at 0, 1, 8, 9, 32, 255, 256, 257, 300, and 1000 root links.
  4. One resolver for both group styles. Rather than teach the old-style symbol-table path about soft links as a special case, GroupRef + resolve_path_target now walk a path through groups of either style identically — so a soft link resolves the same way whichever style holds it, including a value that’s relative to the group holding the link rather than to the file root.
  5. The layout-v4 decoder was rewritten against the specification, not against the fixed-array files it had been empirically derived from: chunk dimensions are now read at their declared encoded width instead of assumed to be one byte, and the indexing-type information block preceding the index address is sized per index type (0 bytes for implicit, 12 for a filtered single chunk, 1/5/6 for fixed-array/extensible-array/v2-B-tree) instead of assumed uniform — which is what made extensible-array and B-tree-v2 datasets read nonsense addresses before.

Getting Started

cargo add oxih5

use oxih5::FileWriter;

let path = std::env::temp_dir().join("out.h5");

// New in 0.2.1: nested groups, DEFLATE compression, and a tiled unlimited dataset.
let mut w = FileWriter::new();
w.write_dataset_f32("/sensors/lab1/signal", &[1.0f32, 2.0, 3.0, 4.0], &[4])?;
w.set_deflate("/sensors/lab1/signal", 6)?;
w.write_string_attr("/sensors/lab1", "units", "volts")?;
w.build(&path)?;

// New in 0.2.1: patch an existing file's dataset bytes in place —
// e.g. a MATLAB .mat file — without touching anything else in it.
let bytes: Vec<u8> = [9.0f64, 9.0, 9.0].iter().flat_map(|v| v.to_le_bytes()).collect();
oxih5::write_dataset_in_place(&path.with_extension("mat"), "/results/values", &bytes).ok();
# Ok::<(), oxih5::OxiH5Error>(())

What’s New in 0.2.1

Tips

This is the foundation

OxiH5 belongs to NoFFI — the COOLJAPAN initiative retiring every C/C++/Fortran/-sys FFI dependency in the Rust ecosystem with a clean, memory-safe, Pure-Rust implementation. It already has real consumers pinned at 0.2.1 across the workspace:

Repository: https://github.com/cool-japan/oxih5

Star the repo ⭐ if you want an HDF5 writer that doesn’t just accept your data — it produces files the reference implementation can actually open.

The era of a Pure-Rust writer whose own output only its own reader could open is over. Pure Rust HDF5 is here — and as of 0.2.1, libhdf5 agrees.

— KitaSan at COOLJAPAN OÜ July 21, 2026

↑ Back to all posts