16 posts
TrustformeRS 0.1.2 patch — full Kubernetes/ACI/OpenShift deploy manifests, EnhancedProfiler exports (Flamegraph/OTLP/Jaeger), scaled-dot-product attention kernel fusion, NMS + token-classification pipelines, Q2_K/Q3_K GGUF quant; the stub-only TPU backend is removed for honesty.
OxiBonsai 0.2.2 adds `oxibonsai repl`: a resident ImageSession that loads the DiT, VAE, and text encoder once and iterates on prompts without re-paying the load/dequant cost — with images shown inline in Ghostty via a pure-Rust kitty graphics protocol, a `:fast`/`:hq` preview→finalize loop, and documented per-platform GPU flags. Sub-2-bit Pure Rust sovereign AI inference for the COOLJAPAN ecosystem.
A quality-of-life and correctness release for OxiBonsai: optimized test/dev compile profiles turn minutes-long numeric tests fast while keeping float parity bit-stable, a VAE precheck fix that finally accepts a .safetensors file, and corrected HuggingFace asset paths. Sub-2-bit Pure Rust sovereign AI inference for the COOLJAPAN ecosystem.
OxiBonsai 0.2.0 opens the 0.2 series: a concurrent engine pool that shares one 1.16 GB embedding table across replicas, a CPU↔Metal byte-identical parity guard, a parity-first CUDA imagen backend (~3.2× to ~31.7s on A4000), --seed byte-exact reproducible images, and a stable-toolchain build — sub-2-bit Pure Rust sovereign AI inference for the COOLJAPAN ecosystem.
OxiBonsai 0.1.5 adds the oxibonsai-image crate — the first pure-Rust, zero-FFI, C/C++/Fortran-free FLUX.2-Klein text-to-image pipeline (DiT + VAE + Qwen3-4B text encoder), parity-validated at cos≥0.999, with Metal flash-attention and ~52–62s end-to-end on an M3. Sub-2-bit Pure Rust sovereign AI inference for the COOLJAPAN ecosystem, now spanning text and image.
OxiBonsai 0.1.4 makes Pure Rust sub-2-bit inference production-grade for serving: adaptive KV-cache compression and adaptive speculative decoding that self-tune under load, full Prometheus observability with per-request X-Request-ID tracing, new FP8 and K-quant GGUF model support, and grammar-constrained decoding for guaranteed-valid JSON — sovereign AI inference for the COOLJAPAN ecosystem.
OxiBonsai 0.1.3 makes sub-2-bit serving smarter: a prefix-cache-aware engine that reuses KV-cache across requests with byte-identical cold/warm parity, runtime tokenizer auto-detection, and a GPU weight cache that uploads once. Sub-2-bit Pure Rust sovereign AI inference for the COOLJAPAN ecosystem.
TrustformeRS 0.1.1 patch — 22 new transformer architectures (49+ total: Falcon2, Gemma2, Mamba2, Qwen2.5, Phi4, Whisper, StarCoder2, xLSTM, and more). ONNX Runtime replaced by Pure Rust oxionnx, tar replaced by oxiarc-archive, Kafka feature-gated. Load any architecture through one AutoModel call — no Python, no PyTorch, no C++ ONNX runtime.
OxiBonsai 0.1.2 adds ONNX ingestion: pull an onnx-community Ternary ONNX release (MatMulNBits, bits=2) and repack it straight to OxiBonsai's GGUF TQ2_0_g128 with a single command — driven by the pure-Rust oxionnx-proto reader, no Python and no onnxruntime. Sub-2-bit sovereign AI inference for the COOLJAPAN ecosystem.
Five days after its 1-bit debut, OxiBonsai grows GPUs: a native CUDA NVRTC backend (~21.9 tok/s on Ternary-Bonsai-1.7B, RTX 3060) and a fused Metal full-forward path (~50 tok/s, ~13x speedup) — plus the new ternary TQ2_0_g128 quant family, with NEON/AVX2/AVX-512 GEMV so it flies on CPU too. Sub-2-bit Pure Rust sovereign AI inference for the COOLJAPAN ecosystem, still with no llama.cpp, no BLAS, no C/Fortran.
Complete GGUF loading + 25 quantized formats + OpenAI-compatible API server — all in pure Rust. 56.2k SLoC, 11 crates, no C/C++/Fortran, built on SciRS2/OxiBLAS/OxiFFT. ≥80% of llama.cpp throughput, WASM/GPU/Python bindings, LLaMA/Mistral/Gemma/Phi/LLaVA support. The sovereign LLM inference layer for SciRS2 and the entire COOLJAPAN ecosystem (now 21M+ SLoC total).
An 8B-parameter language model at roughly 1 bit per weight, running from a single static Rust binary with no llama.cpp, no BLAS, no C/C++/Fortran. OxiBonsai 0.1.0 debuts sub-2-bit Pure Rust sovereign AI inference for the COOLJAPAN ecosystem — SIMD-accelerated, Rayon-parallel, and OpenAI-compatible out of the box.
Pure Rust Hugging Face Transformers — the first stable release. 27+ transformer architectures, 17x CPU BLAS acceleration, GGUF/AWQ/GPTQ quantization, and WASM + server + mobile deployment. No Python, no PyTorch.
OxiRAG 0.1.1 lands production OxiZ SMT verification for Layer 3, real Candle Phi-2/Phi-3 inference and LoRA training, and 5.6x–9.0x faster cosine similarity — all in a Pure Rust RAG engine backed by 1,472 tests.
OxiRAG 0.1.0 is the first release of a Pure Rust Retrieval-Augmented Generation engine: a four-layer pipeline (semantic search, draft verification, SMT logic checking, and knowledge-graph retrieval) that compiles to a single static binary or WASM.
OxiFY 0.1.0 is a Pure Rust, Dify-class LLM workflow orchestration platform: compose AI applications as type-safe directed acyclic graphs. 13 specialized crates, 15+ node types, in-process vector search, CeleRS-backed distribution, 2520+ tests. No Python, no Node.