A file system is not really interplanetary until the planets can talk to each other.
Today we released IPFRS 0.2.0 — the Network Release that turns local-first IPFRS into a fully distributed InterPlanetary File System node, with real libp2p P2P networking, a Kademlia DHT, Bitswap and TensorSwap block exchange, and a Semantic DHT.
No C. No Go. No Fortran. The IPFS world still runs on go-ipfs / Kubo and js-ipfs, on libp2p stacks written in Go and C, on daemons you bolt onto your infrastructure. IPFRS reimplements that whole networking story in Pure Rust — QUIC, Noise, Kademlia, Bitswap, GossipSub — and still compiles to a single static binary (or WASM) that runs everywhere, from a datacenter to a phone.
Why IPFRS 0.2.0 is a game changer
In 0.1.0, IPFRS was an excellent local store — but it was an island. The classic IPFS answer to “make it distributed” is a heavyweight Go daemon and a static file warehouse that knows hashes and nothing else. 0.2.0 is the release where IPFRS stops being local-first and becomes a peer in a real network — without giving up the semantics that make it different.
The concrete wins:
- Real P2P over libp2p with QUIC. QUIC is the primary transport with sub-millisecond connection establishment and multiplexed streams; TCP is the fallback. Noise gives authenticated, encrypted channels; Yamux multiplexes; Ed25519 keypairs mint persistent Peer IDs; AutoNAT detects your NAT type and discovers your external address.
- Content discovery via a Kademlia DHT. Adds auto-provide to the DHT;
get()falls back to the DHT on a local miss;find_providers()now returns a realVec<PeerId>(it returned()before); provider records carry TTLs viaSTART_PROVIDING/STOP_PROVIDING, found through iterative XOR-metric closest-peer lookups. - Two block-exchange protocols. IPFS-compatible Bitswap (
/ipfs/bitswap/1.2.0) with want-list dedup, broadcasting, and session-based transfer; plus TensorSwap (/ipfrs/tensorswap/1.0.0), a custom binary protocol for streaming tensors and ML models with chunk-level resumable transfers. - 10x faster model loading. Apache Arrow IPC rides over TensorSwap with zero-copy deserialization — Arrow buffers are mmap’d straight into Rust slices — benchmarked at 10x faster than JSON/CBOR.
- A Semantic DHT. The HNSW vector index is wired into the Kademlia routing table, so lookups can prefer semantically similar peers, not just XOR-close ones.
Technical Deep Dive: the networking stack
The headline of 0.2.0 is everything between nodes.
1. The libp2p transport layer. QUIC first (multiplexed, sub-millisecond handshakes), TCP as fallback, Noise for authenticated encryption, Yamux for stream multiplexing, Ed25519 for persistent Peer IDs, plus a connection manager, AutoNAT, and the Identify protocol. This is a complete Pure Rust libp2p stack.
2. The Kademlia DHT. DHT bootstrap from well-known peers; auto-provide on add_bytes() / add_file(); DHT fallback on a get() miss; provider records with TTL; iterative closest-peer lookups over the XOR metric. Content discovery is now a network property, not a local lookup.
3. Block and tensor exchange. Bitswap (/ipfs/bitswap/1.2.0) handles general blocks: want-list management with dedup, block broadcasting, session-based transfer with timeout and retry, and bandwidth-aware peer scoring. TensorSwap (/ipfrs/tensorswap/1.0.0) handles ML payloads: chunked streaming with flow control, resume from the last ACK’d chunk, and metadata framing (dtype, shape, strides) in the header. GossipSub (/meshsub/1.1.0) carries block-announcement topics and content-routing events through the mesh, with a fan-out cache for high-churn dedup.
4. The Semantic DHT. HNSW search integrated with Kademlia routing: VectorAnnotatedRecord provider records carry embeddings, proximity-aware lookups prefer semantically similar peers, embeddings propagate across the network, and put_with_vector() / search_similar() work over the DHT. get_routing_convergence() reports health; efficient_partial_sync() gossips only the changed embedding regions; SemanticDhtMetrics and TTL eviction keep it bounded.
5. The modular node. The old single 2,477-line node.rs is now a node/ directory: node/core.rs (lifecycle, config, startup, shutdown), node/network.rs (the swarm event loop and peer management), node/dht.rs (Kademlia ops), node/exchange.rs (Bitswap sessions), and node/health.rs.
Getting Started
Build from source and install the CLI:
git clone https://github.com/cool-japan/ipfrs
cd ipfrs
cargo install --path crates/ipfrs-cli
Now join the network and let the DHT do the work:
ipfrs bootstrap # join the IPFS DHT
CID=$(ipfrs add bigmodel.safetensors) # auto-announced to the DHT
ipfrs swarm peers
ipfrs dht findprovs $CID
The same flow from Rust, with the DHT handling discovery:
use ipfrs::{Node, NodeConfig};
#[tokio::main]
async fn main() -> ipfrs::Result<()> {
let mut node = Node::new(NodeConfig::default())?;
node.start().await?;
// add_bytes auto-provides the CID to the Kademlia DHT
let cid = node.add_bytes(b"Hello, distributed IPFRS!").await?;
// find_providers now returns real peers across the network
let providers = node.find_providers(&cid).await?;
println!("{} provider(s) for {}", providers.len(), cid);
Ok(())
}
What’s New in 0.2.0
- Distributed P2P networking — full libp2p stack with QUIC (+TCP fallback), Noise, Yamux, Ed25519 Peer IDs, AutoNAT, and Identify.
- Kademlia DHT — bootstrap, auto-provide on add, DHT fallback on get, TTL provider records, real
find_providers(). - Bitswap + TensorSwap — IPFS-compatible block exchange plus a resumable binary protocol for tensors and ML models.
- Apache Arrow zero-copy tensors — Arrow IPC over TensorSwap with mmap-direct deserialization, 10x faster model loads.
- Semantic DHT — vector-annotated provider records, proximity-aware lookups, embedding propagation, partial sync, metrics.
- GossipSub pub/sub —
/meshsub/1.1.0block-announcement topics and content-routing events with a fan-out dedup cache. - Distributed reasoning — 0.1.0’s
infer()placeholder is realized: TensorLogic IR serializes to IPLD (DAG-CBOR), rules become content-addressed CIDs with cross-node dedup, sharing rides Bitswap (publish_rule/fetch_rule/import_rules_from_cids), anddistributed_infer()aggregates responses over GossipSub. - Gradient computation graphs —
ComputationGraphStore: a CID-linked DAG with topological ordering, provenance chains, andGradientCheckpointatomic checkpoint/resume. - Robust daemon — background mode (
--background/-d, fork-and-detach), adaptive polling for mobile/IoT, a PID file at/tmp/ipfrs.pid,ipfrs daemon stopgraceful shutdown, 5-category health checks (storage, network, DHT, semantic index, tensorlogic) atGET /health, automatic subsystem recovery, and Prometheus metrics. - OxiARC block compression — per-block zstd/lz4/snappy with magic-byte codec framing via a
CompressionBlockStore<S>wrapper; blocks under 256 bytes stay raw; decompression is transparent. - Persistent indexes — HNSW serializes to disk through the
IndexPersistencetrait (auto-save onnode.stop(), restore onnode.start()), with incremental snapshots (save_smart()delta vs. full when under 10% dirty,load_index_with_delta()). TensorLogic snapshots to IPLD on stop and restores on start. - Temporal pattern matching —
TemporalPatternMatcher, an NFA-based event-sequence matcher with configurable time windows and wildcards (MatcherConfig). - Node.js bindings — NAPI-RS, async over
tokio::sync::Mutex, published as@cool-japan/ipfrs-node. - At-rest encryption — AES-256-GCM / ChaCha20-Poly1305 behind
--features encryption(off by default to keep the Pure Rust default clean). - New network CLI —
ipfrs swarm peers/connect/addrs,ipfrs dht findprovs/provide/findpeer,ipfrs bootstrap,ipfrs gc [--dry-run] [--min-age N], plus advanced queries:ipfrs semantic query "<text>" [--top-k N],ipfrs logic query "<goal>" [--format json|text] [--timeout-ms N],ipfrs query --hybrid --logic "<goal>". Progress bars for files over 10MB; offline detection under 2s. - ~7,200 lines of production Rust across 9 crates. Status: Production Ready (Distributed P2P).
Tips
- Bootstrap before you query the network. Run
ipfrs bootstrapto join the DHT before anyipfrs dht findprovs <cid>oripfrs dht provide <cid>— otherwise discovery has no peers to walk. - Stream models, do not serialize them twice. Pair TensorSwap with Apache Arrow IPC for zero-copy, 10x-faster model loads — buffers mmap straight into Rust slices instead of round-tripping through JSON/CBOR.
- Encrypt at rest when you need it. Build with
--features encryptionfor AES-256-GCM / ChaCha20-Poly1305; it is off by default so the standard build stays 100% Pure Rust. - Save deltas, not whole graphs. The HNSW index uses
save_smart()to write a delta snapshot when under 10% of vectors are dirty, falling back to a full save otherwise — cheap, frequent persistence for large indexes. - Dry-run your garbage collection.
ipfrs gc --dry-run [--min-age N]shows theOrphanGarbageCollectorwhat it would reclaim before you let it delete anything. - Resume, do not restart, big transfers. TensorSwap resumes from the last ACK’d chunk, so an interrupted model pull picks up where it left off instead of starting over.
This is the backbone
IPFRS is now the distributed, content-addressed storage and DHT backbone of the COOLJAPAN ecosystem. It compresses blocks with OxiARC, serializes with OxiCode (oxicode), and streams tensors zero-copy via Apache Arrow — which makes it a natural place to serve model weights for the broader Pure Rust ML stack, from ToRSh and TenfloweRS to OxiMedia, over the same Arrow-on-TensorSwap path.
Repository: https://github.com/cool-japan/ipfrs
Star the repo if a Pure Rust IPFS node — real P2P, a real DHT, and semantics built in — is the distributed foundation you want to build on. Pure Rust distributed storage is here — fast, safe, and sovereign.
— KitaSan at COOLJAPAN OÜ June 16, 2026