The distributed task queue foundation of the COOLJAPAN ecosystem just reached full production readiness.
Today we released CeleRS 0.2.0 — a complete, production-grade pure Rust distributed task queue library that delivers binary-level protocol compatibility with Python Celery while being dramatically faster, type-safe, and enterprise-ready.
No Python. No GIL. No fragile worker processes.
No C/C++ dependencies. No runtime surprises.
Just clean, memory-safe, high-throughput task execution that compiles to a single static binary and runs everywhere — from laptops to edge servers to massive cloud clusters.
Why CeleRS 0.2.0 is a game changer
For years, reliable distributed task queues in enterprise environments meant depending on Python Celery — powerful but plagued by interpreter overhead, memory leaks, and limited type safety.
These tools are powerful but suffer from:
- Python GIL and interpreter bottlenecks
- Complex deployment and debugging
- Lack of compile-time task signature verification
- High memory usage and poor observability
- Difficulty achieving true production-grade reliability at scale
CeleRS 0.2.0 ends all of that.
It delivers 10× throughput compared to Python Celery while being 100% memory-safe and fully type-checked at compile time.
Notable results:
- Target throughput: 10,000 tasks/sec per worker
- Enqueue/dequeue P95 latency: < 10 ms
- Baseline memory footprint: < 50 MB per worker
- 99.99% task delivery guarantee with full observability
Technical Deep Dive: How We Built a Production-Grade Celery Replacement in Pure Rust
CeleRS follows a clean, layered architecture inspired by Celery but rebuilt from the ground up for modern Rust:
-
Core & Protocol Layer (
celers,celers-core,celers-protocol,celers-kombu)
Full binary-compatible Celery v2/v5 wire protocol + procedural macros (#[celers::task]) for compile-time verified task signatures. -
Broker Support (5 high-performance backends)
Redis (Lua + pipelining), PostgreSQL (ACID-safeFOR UPDATE SKIP LOCKED), MySQL (batch ops), RabbitMQ (AMQP exchanges), AWS SQS (long polling). -
Result Backends (3 options)
Redis (TTL + chord sync), Database (PostgreSQL/MySQL with analytics), gRPC (microservices-native). -
Workflow & Scheduling (
celers-canvas,celers-beat)
Full Canvas primitives: Chain, Group, Chord, Map, Starmap, Signature. Periodic tasks via cron, interval, and solar triggers. -
Enterprise Features
Priority queues, Dead Letter Queue (DLQ) with inspection/replay, task cancellation via Pub/Sub, exponential backoff retries, timeout enforcement, graceful shutdown, and event persistence.
Key Rust advantages:
- 18 crates (100% complete and stable)
- Zero-cost abstractions and fearless concurrency
- Full Prometheus + OpenTelemetry observability (pre-built Grafana dashboards)
- Kubernetes-ready health checks and CLI tooling (
celers worker,celers dlq,celers status)
What’s inside 0.2.0 (released March 28)
- All 18 crates marked stable with zero warnings or errors
- Full implementation of Phases 1–9 (compression, distributed locks, event persistence, chunking, heartbeats)
- Enhanced v2/v5 protocol support and CLI tooling
- Production readiness confirmed through 4,075 passing tests across all crates
- Comprehensive examples, architecture decision records, and benchmark suite
This is the foundation
CeleRS is now the official distributed task queue backend for the entire COOLJAPAN stack (total ecosystem: 26M+ SLoC Rust, 660+ crates, 40+ production-grade libraries):
- SciRS2 / NumRS2 — scientific workflow orchestration and batch processing
- OxiMedia / VoiRS — media transcoding and voice synthesis pipelines
- OxiGDAL — geospatial data processing jobs
- Spintronics / OxiHuman — physics simulation and biomechanical modeling tasks
- OxiLLaMa / OxiONNX — LLM and model inference jobs
- RusMES — mail processing and archival workflows
- Future integration with OxiLean for formally verified task routing
Repository: https://github.com/cool-japan/celers
Star the repo if you want a type-safe, high-performance task queue without Python’s overhead.
The era of “just run Celery workers in Python” is over.
Pure Rust distributed task queues — binary compatible, 10× faster, and fully sovereign — are here.
— KitaSan at COOLJAPAN OÜ March 28, 2026