rs3gw 0.2.0 Released — Streaming PUT Backpressure, Graceful Shutdown, and the New rs3ctl CLI

Your object gateway should not fall over the moment someone uploads a terabyte.

Today we released rs3gw 0.2.0 — the production-hardening release that makes large-object PUTs stream instead of buffer, drains in-flight writes on shutdown, and ships a first-class rs3ctl admin CLI.

No C. No C++. No Go. No Fortran. No garbage collector. MinIO is Go. Ceph RGW is C++. rs3gw is a single static Rust binary — the kind of thing that fits in a 50MB container, starts in milliseconds, and gives you auditable, sovereign object storage with no opaque runtime sitting between your data and the kernel. 0.2.0 is about making that binary behave correctly under real load.

Why rs3gw 0.2.0 is a major step

If you have operated MinIO or Ceph RGW, you know the failure modes. A client streams a multi-gigabyte object and resident memory climbs in lockstep because the request body is buffered before it is written. A deploy rolls and in-flight writes get severed mid-flight. Abandoned multipart uploads quietly pile up and eat disk until someone notices. 0.2.0 closes those gaps directly.

• Streaming PUT with backpressure. The PUT handler now streams the request body via Body instead of buffering the whole payload into Bytes, with incremental SHA-256 and MD5 computed as the bytes flow. Memory stays flat whether the object is 4KB or 4TB, and backpressure propagates from the storage layer back to the socket.
• Graceful shutdown with in-flight drain. On termination, rs3gw stops accepting new work and drains outstanding requests with a 30s timeout, so a rolling restart no longer truncates active writes.
• Background multipart GC scheduler. Abandoned multipart uploads are reclaimed automatically on a configurable retention/interval, instead of accumulating until disk fills.
• SigV4 authentication with rate limiting. Failed-auth attempts are throttled at 10 failures per 60s, blunting credential-stuffing against the gateway.
• Content-Length audit on every response. GET, HEAD, and range responses now verify their declared Content-Length against what is actually served — no silently truncated downloads.
• gRPC parallel delete and range GET. The gRPC surface gains buffer_unordered(10) concurrent deletes and proper range GET via range_start/range_end fields.

Technical Deep Dive

The streaming PUT path. The headline change runs through src/api/ and src/storage/. Where 0.1.0 collected the full body into Bytes before handing it to the storage engine, the 0.2.0 PUT handler streams via Body, hashing incrementally so the SHA-256 and MD5 are finalized exactly when the last chunk lands — no second pass over the payload. StorageError::Io deliberately no longer auto-derives From; conversions are now an explicit, audited mapping so an I/O fault turns into the correct S3 error rather than a generic 500. Compression is applied at the storage layer (src/storage/) as the stream is written, and the new metrics record original vs. compressed bytes and the achieved ratio.

Lifecycle and safety. Request lifecycle is tracked through InFlightTracker and InFlightGuard: every request takes a guard on entry and releases it on completion, which is what makes the 30s graceful drain precise rather than best-effort. The background multipart GC scheduler runs on its own configurable cadence. Filesystem faults are mapped to the right S3 semantics — PermissionDenied becomes AccessDenied, StorageFull becomes InsufficientStorage — so clients see actionable, correct status codes.

gRPC enhancements. In src/grpc/ (server.rs, object.rs), batch deletes now fan out with buffer_unordered(10) for real concurrency instead of serial round-trips, and range GET is wired end to end through range_start/range_end. DeleteObjects also handles partial failures correctly, reporting per-key results rather than failing the whole batch.

Auth hardening and health. SigV4 lives in src/auth/v4.rs, now with the failure rate limiter. For orchestration, 0.2.0 exposes a /health JSON endpoint for liveness and a /ready probe for readiness, so Kubernetes can distinguish “process alive” from “ready to serve.”

A note on compression: 0.2.0 still uses Zstd and LZ4 (with Flate2 available) directly. What is new is the metrics wiring around them — you can now see the compression ratio and byte counts rather than guessing.

Getting Started

rs3gw is a server, so you build and run it rather than adding it as a library.

git clone https://github.com/cool-japan/rs3gw.git
cd rs3gw
cargo build --release
./target/release/rs3gw

Configure it through environment variables:

export RS3GW_BIND_ADDR="0.0.0.0:9000"
export RS3GW_STORAGE_ROOT="./data"
export RS3GW_ACCESS_KEY="minioadmin"
export RS3GW_SECRET_KEY="minioadmin"
export RS3GW_COMPRESSION="zstd:3"

Then point any S3 client at it — it listens on :9000:

aws --endpoint-url http://localhost:9000 s3 mb s3://my-bucket
aws --endpoint-url http://localhost:9000 s3 cp big.dat s3://my-bucket/   # exercises new PUT streaming + multipart GC

And drive the gateway from the new rs3ctl admin CLI:

./target/release/rs3ctl health
./target/release/rs3ctl metrics
./target/release/rs3ctl gc-multipart

What’s New in 0.2.0

Streaming and safety

• PUT body streamed via Body instead of buffered Bytes, with incremental SHA-256/MD5 hashing.
• Graceful shutdown that drains in-flight requests (30s timeout).
• Background multipart GC scheduler with configurable retention/interval.
• InFlightTracker / InFlightGuard for request lifecycle tracking.
• Filesystem error mapping (PermissionDenied → AccessDenied, StorageFull → InsufficientStorage).
• Content-Length audit and verification on all GET/HEAD/range responses.
• Compression applied at the storage layer with ratio and byte-count metrics (Zstd/LZ4).
• DeleteObjects partial-failure handling.

gRPC

• Parallel delete with buffer_unordered(10) concurrency.
• Range GET support via range_start/range_end fields.

Auth and health

• SigV4 authentication with rate limiting (10 failures / 60s).
• /health JSON endpoint and /ready probe.

Analytics and Select

• Predictive analytics and cost forecasting.
• S3 Select with query plan cache and result cache.
• Arrow Flight protocol support.
• Object Lambda transformations.
• Data deduplication with content-addressable storage.
• Intelligent tiering with storage class transitions.
• Advanced replication manager.

Tooling and docs

• New rs3ctl CLI: health, metrics, gc-multipart, benchmark, diagnose.
• Production deployment guide (sizing, env vars, TLS, troubleshooting, monitoring).
• README with quick start, boto3 / AWS CLI examples, and an API compatibility table.
• OpenTelemetry environment-variable documentation.
• Bucket stubs module documentation covering all 53 stub endpoints.
• Smoke tests, zero-byte roundtrip tests, and path-traversal rejection regression tests.

Tips

• Large uploads now stream — no memory tuning required. Whether an object is megabytes or terabytes, the PUT path holds memory flat. You no longer need to size the gateway around your biggest expected upload.
• Reclaim abandoned multipart uploads. Run ./target/release/rs3ctl gc-multipart on demand, or just let the background GC scheduler handle it on its configured retention/interval.
• Expect 403s under credential stuffing. SigV4 rate limiting trips after 10 auth failures in 60s. If a misconfigured client is hammering bad credentials, that is the limiter doing its job.
• Set compression and watch the metrics. Use RS3GW_COMPRESSION="zstd:3" and read the new ratio / original-vs-compressed byte metrics to confirm you are actually saving space on your workload.
• Wire health and readiness separately in K8s. Point liveness at /health and readiness at /ready so a pod that is up but still warming does not receive traffic prematurely.
• Lean on the richer gRPC surface. Clients can now request byte ranges (range_start/range_end) and batch-delete in parallel — useful for high-fan-out cleanup and partial reads.

This is the foundation

rs3gw is built on scirs2-io, the I/O layer of the SciRS2 scientific-computing stack — that is what backs the storage engine and ties the gateway into the broader COOLJAPAN ecosystem alongside siblings like NumRS2, OptiRS, PandRS, OxiBLAS, and OxiCode. The goal is a coherent, Pure Rust foundation where your numerical stack and your object store speak the same language and ship the same way: as auditable static binaries with no foreign runtime underneath.

Repository: https://github.com/cool-japan/rs3gw

Star the repo if you want S3 you can actually read, audit, and own. Sovereign storage, one binary, no surprises.

— KitaSan at COOLJAPAN OÜ March 16, 2026