14 posts
OxiONNX 0.1.7 grows oxionnx-cuda from 25 to 40 accelerated ops with real Conv support, session-lifetime weight/buffer residency, and batched MatMul up to 17x faster — plus a fix for a two-stream race that silently returned wrong GEMM results on real hardware. 3,595 tests passing. The sovereign Pure Rust ONNX layer for COOLJAPAN.
OxiONNX 0.1.6 ships GPU weight/activation residency and a direct implicit-GEMM Conv2D kernel (~692 GFLOP/s on M3) — plus the fix making wasm32 CPU inference actually run, and a CoreML compile cache ending 857 GB of orphaned recompiles. 3,212 tests passing. The sovereign Pure Rust ONNX layer for COOLJAPAN.
OxiONNX 0.1.5 adds async execution, cancellation, streaming generation, and session serialization — backed by a 12-lens, 232-finding hardening pass across parsing, spec-conformance, and GPU/CUDA/DirectML dispatch. 23 new operators, 2,946 tests passing. The sovereign Pure Rust ONNX layer for COOLJAPAN.
OxiBonsai 0.2.3 ships a two-pass, 109-agent production-hardening audit: a P0 fix for a silent CUDA prefill/decode KV-cache desync, full Metal GPU coverage for Q4_0/Q8_0/K-quant and FP8 batch prefill, real OpenAI-compatible penalties and logprobs, and tokenizer fidelity fixes for CJK/emoji text — 91 confirmed findings closed, 5,158 tests passing. Sub-2-bit Pure Rust sovereign AI inference for the COOLJAPAN ecosystem.
OxiBonsai 0.2.2 adds `oxibonsai repl`: a resident ImageSession that loads the DiT, VAE, and text encoder once and iterates on prompts without re-paying the load/dequant cost — with images shown inline in Ghostty via a pure-Rust kitty graphics protocol, a `:fast`/`:hq` preview→finalize loop, and documented per-platform GPU flags. Sub-2-bit Pure Rust sovereign AI inference for the COOLJAPAN ecosystem.
A quality-of-life and correctness release for OxiBonsai: optimized test/dev compile profiles turn minutes-long numeric tests fast while keeping float parity bit-stable, a VAE precheck fix that finally accepts a .safetensors file, and corrected HuggingFace asset paths. Sub-2-bit Pure Rust sovereign AI inference for the COOLJAPAN ecosystem.
OxiBonsai 0.2.0 opens the 0.2 series: a concurrent engine pool that shares one 1.16 GB embedding table across replicas, a CPU↔Metal byte-identical parity guard, a parity-first CUDA imagen backend (~3.2× to ~31.7s on A4000), --seed byte-exact reproducible images, and a stable-toolchain build — sub-2-bit Pure Rust sovereign AI inference for the COOLJAPAN ecosystem.
OxiBonsai 0.1.5 adds the oxibonsai-image crate — the first pure-Rust, zero-FFI, C/C++/Fortran-free FLUX.2-Klein text-to-image pipeline (DiT + VAE + Qwen3-4B text encoder), parity-validated at cos≥0.999, with Metal flash-attention and ~52–62s end-to-end on an M3. Sub-2-bit Pure Rust sovereign AI inference for the COOLJAPAN ecosystem, now spanning text and image.
OxiBonsai 0.1.4 makes Pure Rust sub-2-bit inference production-grade for serving: adaptive KV-cache compression and adaptive speculative decoding that self-tune under load, full Prometheus observability with per-request X-Request-ID tracing, new FP8 and K-quant GGUF model support, and grammar-constrained decoding for guaranteed-valid JSON — sovereign AI inference for the COOLJAPAN ecosystem.
OxiBonsai 0.1.3 makes sub-2-bit serving smarter: a prefix-cache-aware engine that reuses KV-cache across requests with byte-identical cold/warm parity, runtime tokenizer auto-detection, and a GPU weight cache that uploads once. Sub-2-bit Pure Rust sovereign AI inference for the COOLJAPAN ecosystem.
OxiBonsai 0.1.2 adds ONNX ingestion: pull an onnx-community Ternary ONNX release (MatMulNBits, bits=2) and repack it straight to OxiBonsai's GGUF TQ2_0_g128 with a single command — driven by the pure-Rust oxionnx-proto reader, no Python and no onnxruntime. Sub-2-bit sovereign AI inference for the COOLJAPAN ecosystem.
Five days after its 1-bit debut, OxiBonsai grows GPUs: a native CUDA NVRTC backend (~21.9 tok/s on Ternary-Bonsai-1.7B, RTX 3060) and a fused Metal full-forward path (~50 tok/s, ~13x speedup) — plus the new ternary TQ2_0_g128 quant family, with NEON/AVX2/AVX-512 GEMV so it flies on CPU too. Sub-2-bit Pure Rust sovereign AI inference for the COOLJAPAN ecosystem, still with no llama.cpp, no BLAS, no C/Fortran.
An 8B-parameter language model at roughly 1 bit per weight, running from a single static Rust binary with no llama.cpp, no BLAS, no C/C++/Fortran. OxiBonsai 0.1.0 debuts sub-2-bit Pure Rust sovereign AI inference for the COOLJAPAN ecosystem — SIMD-accelerated, Rayon-parallel, and OpenAI-compatible out of the box.
High-performance ONNX runtime written entirely in pure Rust. Zero C/C++ dependencies, 147 operators fully supported, wgpu GPU acceleration, SIMD (AVX2/NEON), WASM + no_std ready, graph optimizer, async execution, model encryption. 30k+ SLoC, 590+ tests. The sovereign ONNX inference layer for SciRS2 and the entire COOLJAPAN ecosystem (now 21M+ SLoC total).