◉ COOLJAPAN
← All posts
August 13, 2026 • 10 min read

OxiCUDA 0.5.5 Released — The Alt-Backend Audit Finds Metal's GPU Kernels Were Never Called

OxiCUDA 0.5.5 audits oxicuda-metal and oxicuda-webgpu on real Apple Silicon, fixing conv2d/attention/softmax false completions where finished MSL kernels sat unused behind CPU scalar loops. Separately, a downstream investigation wires up an unreachable split-K GEMM path, replaces a silent Winograd no-op with a real kernel, and fixes wrong Ampere GA10x hardware constants — measured on a real RTX A4000. 38,987 tests, ~1.31M SLoC, 74 crates.

release oxicuda cuda gpu pure-rust metal webgpu gemm convolution

A finished GPU kernel sitting in the same file as the function that should call it — but doesn’t — is the kind of bug that no type checker catches and no user-facing error reports. OxiCUDA 0.5.5 found three of them.

Today we released OxiCUDA 0.5.5 — a correctness release built around two adversarial audits. The first runs the alt-backend audit that a prior production-readiness wave queued but never reached: oxicuda-metal and oxicuda-webgpu, audited and fixed on real Apple Silicon (M3, Metal 4, macOS/arm64). The second closes out a downstream investigation triggered by oxiface, a Rust face-swap CLI, underperforming on Linux+NVIDIA relative to its CoreML path — traced all the way to oxicuda-blas/oxicuda-dnn GEMM and convolution dispatch, and verified end to end on a real RTX A4000.

No CUDA SDK. No nvcc. No C/C++ toolchain. OxiCUDA is a type-safe, memory-safe, pure-Rust replacement for the entire NVIDIA CUDA Toolkit software stack — cuBLAS, cuDNN, cuFFT, cuSPARSE, cuSOLVER, cuRAND, and more, across 74 crates and ~1.31M lines of safe Rust. libcuda.so/nvcuda.dll is loaded dynamically at runtime via libloading; nothing about building or running OxiCUDA requires the CUDA SDK, headers, or a C/C++ compiler. The result compiles to a single static binary and runs everywhere the driver is present.

Why OxiCUDA 0.5.5 is a game changer

The headline finding is the kind of bug that only shows up when you go looking for it, not when you run the test suite:

OxiCUDA 0.5.5 ends all of that.

Technical Deep Dive

  1. Metal dispatch layer. A new backend/nn.rs module carries the real conv2d_forward/attention dispatch; six ad-hoc GPU dispatch paths plus the FFT plan now route through a shared commit_and_wait/status_to_result so a GPU-side failure surfaces as Err instead of a silently wrong result. gemm/batched_gemm/gemm_f16 move to a runtime-parameterised v2 kernel family supporting all four transpose combinations with padded leading dimensions.
  2. WebGPU correctness layer. Real adapter limits, a non-fatal uncaptured-error handler, submission-index-scoped readback waits, checked u32 stride/dispatch conversions for batched_gemm, and a per-element bound fix in scan_wgsl’s write stage that could write one element past n on an odd remainder.
  3. GEMM/conv dispatch layer. Split-K’s reduction workspace moved from an alloc-per-call (a device-wide cuMemFree barrier, and incompatible with CUDA graph capture) to a bounded, reusable cache keyed on (stream, output_type, element count). compute_grid now sizes the naive GEMM kernel’s launch from real device occupancy (sm_count * max_threads_per_sm) instead of a CTA-tiling assumption that didn’t match the kernel’s actual grid-stride design — measured 191 → 747 GFLOPS at 1024³ F32 from that fix alone.
  4. Infrastructure layer. A new oxicuda-dnn kernel_cache module turns a repeated kernel-generation call into a hash lookup plus an Arc clone instead of a fresh JIT compile (~194µs per call saved on an RTX A4000), now wired into every kernel-generating module in the crate. A new oxicuda-memory StagingBuffer gives hot-path H2D/D2H transfers a reusable page-locked allocation, measured 1.55x-1.75x H2D and 1.77x-2.67x D2H over the driver’s pageable path.

Getting Started

cargo add oxicuda

The new compute module probes every backend compiled into the build and hands back the best one already initialised — no need to know ahead of time whether a machine has an NVIDIA GPU, an Apple GPU, or neither:

use oxicuda::backend::ComputeBackend;

fn main() -> oxicuda::backend::BackendResult<()> {
    let backend = oxicuda::compute::default_backend()?;
    println!("computing on the {} backend", backend.name());

    let ptr = backend.alloc(1024)?;
    backend.free(ptr)?;
    Ok(())
}

On macOS with the metal feature enabled, this returns a real MetalBackend running on the Apple GPU — the same backend this release’s conv2d/attention/softmax fixes land in. Without a GPU backend compiled in, or where one can’t open, it falls back to CpuBackend rather than erroring:

[dependencies]
oxicuda = { version = "0.5", features = ["metal"] }

To run the on-device validation yourself against a real GPU:

cargo test --features gpu-tests -p oxicuda-metal
cargo test --features gpu-tests -p oxicuda-blas

What’s New in 0.5.5

Tips

This is the foundation

OxiCUDA is the GPU layer beneath the rest of the COOLJAPAN ecosystem. Its architecture diagram puts SciRS2, OxiONNX, TrustformeRS, and ToRSh directly on top of it, with OxiBLAS and OxiFFT rounding out the list of projects that lean on this stack — and the oxicuda crate itself is an umbrella re-export over all 74 crates, so a single cargo add oxicuda pulls in whichever subsystem a dependent project needs. The GEMM/conv fixes in this release were found through exactly that kind of downstream pressure: a real inference pipeline (oxiface) hitting a real performance cliff on real hardware.

Repository: https://github.com/cool-japan/oxicuda

Star the repo if you think “the kernel exists in the codebase” and “the kernel actually runs” deserve to be checked as two separate things. Every star tells us to keep auditing.

The era of trusting a green test suite over an adversarial audit is over. Pure Rust GPU computing is here — and as of 0.5.5, its Metal and WebGPU backends have been checked against real hardware, not just compiled.

— KitaSan at COOLJAPAN OÜ August 13, 2026

↑ Back to all posts