3FS is a distributed file system designed to address the challenges of AI training and inference workloads. It solves the problem of efficiently sharing storage across large-scale AI applications by disaggregating storage across hundreds of nodes equipped with SSDs and RDMA networks. The system implements Chain Replication with Apportioned Queries for strong consistency, allowing applications to access storage in a locality-oblivious manner without learning new APIs—it exposes a standard file interface backed by a transactional key-value store.
The tool is built for organizations running large-scale AI workloads that require high-throughput shared storage for data preparation, training, and inference. It suits projects involving distributed data analytics pipelines, parallel checkpointing for training, random-access dataloaders across compute nodes, and KVCache serving for inference. The disaggregated architecture enables applications to treat thousands of SSDs and hundreds of storage nodes as a unified resource, eliminating the need for data prefetching or shuffling in many scenarios.
Adoption makes sense for teams with infrastructure at scale—the system is demonstrated on clusters with 180 storage nodes and 500+ client nodes, achieving aggregate read throughput in the multi-terabyte-per-second range. It is particularly valuable when you need strong consistency guarantees without sacrificing performance, and when your workload involves managing large volumes of intermediate outputs or serving high-throughput random access patterns. The standard file interface means minimal application code changes compared to custom storage solutions.
The project maintains active development with regular commits across core storage, metadata, and API components. Work spans infrastructure improvements, performance optimization on large clusters, and support for diverse AI workload patterns including data preparation, training checkpointing, and inference serving. The codebase includes comprehensive benchmarking infrastructure and documentation covering design, deployment, and API usage.
