◉ COOLJAPAN
← All posts
August 6, 2026 • 10 min read

OxiH5 0.2.3 Released — Six Writer Gaps Closed, Extensible-Array Chunk Indexes Now Readable

OxiH5 0.2.3 closes six writer roadmap gaps — compound records, vlen sequences, array/opaque/bitfield types, big-endian/half-precision floats, soft/external/hard links — makes extensible-array chunk indexes readable, fixes five crash/OOM classes. 987 tests passing — COOLJAPAN's sovereign scientific-data layer.

release oxih5 pure-rust cooljapan noffi hdf5 netcdf scientific-data fuzzing

A create_dataset(..., chunks=..., maxshape=(None, ...)) file — the ordinary h5py append-able dataset under libver='latest' — used to come back from OxiH5 as a typed NotImplemented. Not anymore.

Today we released OxiH5 0.2.3 — the release that closes six writer roadmap gaps in one pass, teaches the reader the real on-disk shape of an extensible-array chunk index instead of refusing to touch it, and fixes a mis-parsed array datatype plus five crash/OOM classes on crafted input, two of them surfaced by brand-new fuzz targets within seconds of their first run.

No libhdf5. No FFI. No -sys crates. OxiH5 parses and writes the real HDF5 binary format — superblock, object headers, B-trees, heaps, the global heap, the filter pipeline, all eleven datatype classes — using nothing but std.
#![forbid(unsafe_code)] guards oxih5-core; the facade crate carries exactly one documented unsafe block, the mmap call behind open_mmap. It compiles to a single static binary (or WASM) and runs everywhere.

Why OxiH5 0.2.3 is a game changer

Before this release:

OxiH5 0.2.3 ends all of that:

Technical Deep Dive: what changed under the hood

  1. The extensible-array index isn’t self-describing — the client is. The unfiltered client (H5EA_CLS_CHUNK_ID) stores a bare 8-byte chunk address and takes its size from the uncompressed chunk size; the filtered client (H5EA_CLS_FILT_CHUNK_ID) stores the address, a variable-width stored size, and a 4-byte filter mask. A chunk’s position comes from H5VM_array_offset_pre over swizzled coordinates — the unlimited dimension rotated to the front, not swapped with dimension 0 — computed against the dataspace’s maximum dimensions so the numbering survives the dataset growing. Data-block element counts follow the sblk_info recurrence libhdf5 derives from the header’s own creation parameters, and paged data blocks (per-page checksums, page-init bitmasks) are decoded rather than treated as allocator leftovers.
  2. New-style groups need real metadata checksums, not just the right bytes. write/checksum.rs implements the Jenkins lookup3 hash — also H5_checksum_metadata — over exactly the byte ranges the fractal heap header, its root direct block, the B-tree v2 header, and its leaf node cover. Past max_compact (8 members), the group converts from Link Info + Link messages to a fractal heap indexed by that checksum-verified B-tree, and a converted old-style group keeps its empty B-tree/local-heap/SNOD structures with no symbol-table message — the exact shape libhdf5 leaves behind on conversion.
  3. Every integer that reaches a slice or an allocation from on-disk bytes now goes through checked_add/checked_mul. btree_v1_chunk::collect‘s old bounds check (off + 24 > file_data.len()) could itself overflow and silently pass a huge off through to the next slice; parse_hyperslab_v1_blocks used to hand a raw on-disk u32 block count straight to Vec::with_capacity, so a 39-byte crafted VDS mapping block could request a multi-gigabyte allocation before reading a single byte. Both are typed errors now, alongside the vlen/global-heap decoders’ n_elems * 16 multiplications.
  4. chunked.rs (1963 lines) and ea_index.rs (819 lines) both split along their natural seams — chunked/{cache,geometry,index,read,slice,tests}.rs and ea_index/{header,blocks,elements,tests}.rs — with mod.rs glob re-exporting everything that used to be reachable at the old path. Purely internal; every public item and every test keeps its exact address.

Getting Started

cargo add oxih5

use oxih5::FileWriter;

let path = std::env::temp_dir().join("output.h5");
let mut writer = FileWriter::new();

// A 1-D float64 dataset, chunked and gzip-compressed.
let readings: Vec<f64> = (0..1024).map(|i| i as f64 * 0.5).collect();
writer.write_dataset_f64("readings", &readings, &[readings.len()])?;
writer.set_chunking("readings", &[256])?;
writer.set_deflate("readings", 6)?;
writer.write_string_attr("readings", "units", "volts")?;

// Nested groups are created along the way — no separate "create group" call.
let flags: Vec<i32> = (0..12).collect();
writer.write_dataset_i32("instrument/status/flags", &flags, &[3, 4])?;
writer.write_string_attr("instrument/status", "firmware_version", "2.3.1")?;

writer.build(&path)?;

let file = oxih5::open(&path)?;
let ds = file.dataset("readings")?;
println!("read back {} elements, shape {:?}", ds.as_f64()?.len(), ds.shape);
# Ok::<(), oxih5::OxiH5Error>(())

This is trimmed from the runnable crates/oxih5/examples/write_dataset.rs (cargo run -p oxih5 --example write_dataset); read_dataset.rs and oxinetcdf’s write_and_read.rs are the same idea for reading via slices and hyperslabs, and for NetCDF-4 dimensions/variables/attributes.

What’s New in 0.2.3

Tips

This is the foundation

OxiH5 belongs to NoFFI — the COOLJAPAN initiative retiring every C/C++/Fortran/-sys FFI dependency in the Rust ecosystem with a clean, memory-safe, Pure-Rust implementation. Real consumers are pinned at 0.2.3 across the workspace today:

Repository: https://github.com/cool-japan/oxih5

Star the repo ⭐ if a chunk index returning NotImplemented on the default append-able h5py layout shouldn’t have been acceptable for this long.

The era of a Pure-Rust HDF5 writer that could only do scalars and strings is over. Compound records, ragged sequences, real links, and a chunk index that actually decodes — that’s 0.2.3.

— KitaSan at COOLJAPAN OÜ August 6, 2026

↑ Back to all posts