◉ COOLJAPAN
← All posts
July 11, 2026 • 9 min read

OxiWhisper 0.1.2 Released — Speaker Diarization, Word Timestamps, and Translation

OxiWhisper 0.1.2 adds a full speaker-diarization pipeline (VAD, clustering, RTTM/DER export, optional ECAPA-TDNN embeddings), word-level timestamps, a translation task, and temperature-fallback decoding — the sovereign speech-recognition layer for the COOLJAPAN ecosystem, still zero C or Python.

release oxiwhisper whisper speech-to-text asr rust audio transcription speaker-diarization

Knowing what was said is only half the job. OxiWhisper 0.1.2 also answers who said it.

Today we released OxiWhisper 0.1.2 — a feature release that adds a full speaker-diarization pipeline, first-class word-level timestamps, a translation task, and temperature-fallback decoding to the Pure Rust Whisper engine.

The ground rule has not moved: OxiWhisper is still a Pure Rust Whisper inference engine with no C, no C++, and no Python anywhere in the build, compiling to a single static binary — or to WebAssembly — that runs Whisper inference end to end. The incumbents have not moved either. whisper.cpp is still a vendored C++ tree you cross-compile per target. Python + PyTorch still drags in libtorch and an interpreter. And if you wanted speaker labels on top of either one, the standard answer was pyannote.audio — a second Python/PyTorch dependency bolted onto your transcription stack just to know who was talking. 0.1.2 folds that second pipeline into the same Rust crate instead.

Why OxiWhisper 0.1.2

If 0.1.1 was about meeting the formats and hardware people actually have, 0.1.2 is about closing the gap between “Whisper transcribes text” and what a real transcription product needs: who is talking, where each word actually sits in time, and what to do when a decode attempt goes sideways. All three used to mean reaching outside Rust. Not anymore.

ComponentStatusTests
Core inference (encoder/decoder)Stable579 passing
Quantized inference (Q4_0/Q5_0/Q8_0)Stable40+
SIMD kernels (AVX2/NEON/WASM)Stable15+
Streaming APIStable8+
Word timestamps (monotonic peak)Stable15+
ONNX model loadingStable28+
Speaker diarization (VAD, clustering, RTTM/DER)Stable*99+

* The pipeline — VAD, embedding, clustering, resegmentation, RTTM/DER export — is feature-complete and stable. Producing production-accurate speaker labels requires supplying an external speaker-embedding model (e.g. ECAPA-TDNN via ONNX); the built-in baseline embedder is intentionally low-accuracy and meant for tests and no-model demos only. More on this in Tips.

Technical Deep Dive

The core pipeline is unchanged — it is still the real Whisper encoder-decoder transformer:

Audio (WAV/f32) -> Mel Spectrogram (OxiFFT) -> Encoder (Conv + Transformer)
-> Decoder (Autoregressive + KV Cache + Beam Search) -> Tokenizer -> Text

0.1.2 adds a second pipeline alongside it, and deepens the first one at several points.

Diarization is a new stage built on top of the decoder’s output rather than inside it: energy-based VAD finds speech regions, those get sliced into uniform sub-segments, each sub-segment is embedded by a SpeakerEmbedder, cosine-affinity clustering (agglomerative or spectral, both with speaker-count estimation) groups the embeddings, resegmentation cleans up the boundaries, and the result fuses with word timestamps into a SpeakerTranscript. Two embedders ship: WhisperEncoderEmbedder, which mean-pools the audio encoder’s own features and is explicitly a low-accuracy baseline — Whisper’s encoder is trained to be speaker-invariant, not speaker-discriminative — and EcapaOnnx, which loads an external ECAPA-TDNN / x-vector checkpoint through oxionnx for real accuracy. The clustering and the symmetric eigensolver behind spectral clustering are hand-written inline in diarize — the feature pulls in no ndarray, no nalgebra, and deliberately no scirs2-cluster / scirs2-linalg.

Decoding got denser too. transcribe_words() wires the existing, fully-tested dtw.rs (align_tokens_dp_dtw, build_word_segments) into the public API as WordTimedTranscript / WordSegment, working with the greedy and temperature-sampling paths (beam_width > 1 currently returns ConfigError). A new Task enum (Transcribe / Translate) drives the <|translate|> (50358) token for any-language-to-English decoding. fallback_temperatures plus logprob_threshold implement OpenAI-style retry: a low-confidence or degenerate decode gets re-attempted at the next temperature in the schedule, a no-op by default. no_speech_prob is now captured from the raw prefill logits before any suppression across all three samplers, and ApplyTimestampRules — OpenAI’s timestamp-token forcing logic — runs automatically whenever timestamps are requested.

Audio ingestion moved to the Symphonia 0.6 API: decode_with_symphonia now uses GenericAudioBufferRef::copy_to_vec_interleaved in place of SampleBuffer, CodecParameters::Audio / AudioDecoderOptions in place of the old CODEC_TYPE_NULL / DecoderOptions, and Probe::probe() / make_audio_decoder() in place of their predecessors, with next_packet() now handling Ok(None) cleanly at end-of-stream — zero API change for callers of load_audio().

Robustness closed two real bugs. In decode_beam, a stop-token candidate seeded as a zero-token “done” beam could out-score every genuine hypothesis once scores were length-normalized, silently returning empty text for audible input; seeding now over-samples the top-k candidates and filters stop tokens before seeding. And in src/gguf/parse.rs / src/gguf/spec.rs, tensor sizes are now computed with checked arithmetic and validated against the actual file length before allocation, so a truncated or adversarially-crafted .gguf file returns OxiWhisperError::InvalidModel instead of panicking or attempting a multi-gigabyte allocation.

Getting Started

cargo add oxiwhisper

use oxiwhisper::{WhisperModel, TranscribeOptions};
use std::path::Path;

fn main() -> Result<(), oxiwhisper::OxiWhisperError> {
    let model = WhisperModel::from_file(Path::new("ggml-tiny.bin"))?;
    let audio = oxiwhisper::audio::load_wav(Path::new("audio.wav"))?;
    let text = model.transcribe(&audio, &TranscribeOptions::default())?;
    println!("{text}");
    Ok(())
}

That is still the whole basic flow — load a GGML or GGUF checkpoint, read a WAV, transcribe. Diarization, word timestamps, and translation are opt-in from here; see Tips below.

What’s New in 0.1.2

Tips

This is the foundation

OxiWhisper’s place in the COOLJAPAN stack has not moved — it is still the speech-recognition layer, anchored by OxiFFT for the mel front end. What is new this release is how far oxionnx reaches: it used to be only the optional path for loading ONNX Whisper checkpoints, and now it also carries real speaker embeddings into the diarization pipeline via EcapaOnnx. Diarization’s clustering and eigensolver deliberately do not reach for SciRS2’s scirs2-cluster / scirs2-linalg — they are hand-written inline, keeping the feature’s dependency footprint as small as its accuracy claims are honest. SciRS2 and NumRS2 remain the numerical bedrock elsewhere in the ecosystem, and VoiRS stays the natural companion on the text-to-speech side — between the two, a Pure Rust project can now find out who said what and say something back, with no foreign-language runtime anywhere in the loop.

Repository: https://github.com/cool-japan/oxiwhisper

Star the repo if speaker-attributed, word-timestamped Pure Rust transcription — without a second Python toolchain bolted on — is what you have been waiting for.

Pure Rust speech recognition is here — fast, safe, and sovereign.

— KitaSan at COOLJAPAN OÜ July 11, 2026

↑ Back to all posts