2 posts
This AmateRS patch activates horizontal sharding (QueryRouter, ShardRegistry, a placement scheduler that drives shard splits over Raft), adds end-to-end OpenTelemetry tracing, an io_uring WAL, an FHE circuit cache, constant-time API-key auth, and an async Python SDK.
OxiBonsai 0.1.4 makes Pure Rust sub-2-bit inference production-grade for serving: adaptive KV-cache compression and adaptive speculative decoding that self-tune under load, full Prometheus observability with per-request X-Request-ID tracing, new FP8 and K-quant GGUF model support, and grammar-constrained decoding for guaranteed-valid JSON — sovereign AI inference for the COOLJAPAN ecosystem.