A high-performance distributed file system designed to address the challenges of AI training and inference workloads.
10k
Stars
1.1k
Forks
154
Open issues
30
Contributors
AI Analysis
3FS (Fire-Flyer File System) is a high-performance distributed file system purpose-built for AI training and inference workloads, leveraging SSDs and RDMA networks to provide strong consistency and disaggregated storage access. It is specialized for large-scale machine learning infrastructure, data preparation pipelines, and distributed compute clusters—not a general-purpose file system, and primarily benefits organizations running enterprise-scale AI training operations.
Inferred from signals mentioned in the README (tests, CI, type safety) — not a review of the actual code.
AI's overall editorial judgment — not an average of the bars above, can weigh other factors too.
DeepSeek open-sources 3FS, the distributed file system powering large-scale AI training at 6.6 TiB/s aggregate read throughput
3FS (Fire-Flyer File System) is a high-performance distributed file system built by DeepSeek specifically for AI training and inference workloads. It leverages RDMA networking and modern NVMe SSDs to deliver disaggregated storage at scale. Primary users are AI infrastructure teams running large GPU clusters who need high-throughput parallel checkpointing, fast dataloader access, KV-cache offloading for LLM inference, and data preparation pipelines. The system is backed by CRAQ-based strong consistency and FoundationDB-backed metadata, distinguishing it from eventually-consistent alternatives.
Created February 2025 as part of DeepSeek's open-source infrastructure release wave following their high-profile R1 model launch. Represents the internal storage layer used in DeepSeek's own training clusters, now made public under MIT license.
Star growth was almost entirely driven by the DeepSeek halo effect in early 2025, when the company's AI research attracted intense global attention and engineers rushed to examine the full infrastructure stack. Growth has since slowed significantly — only 8 stars in the last 7 days as of June 2026 — indicating interest has stabilized to a niche of specialist infrastructure engineers rather than broad developer curiosity.
Strong indirect evidence: DeepSeek's own published benchmarks show deployment on a 180-node cluster with 500+ client nodes achieving 6.6 TiB/s aggregate read throughput, and GraySort results on a 75-node cluster. This constitutes credible proof of internal production use at scale. Third-party external adoption beyond DeepSeek is not publicly verified as of this analysis.
Appears to implement a disaggregated storage architecture with stateless metadata services backed by FoundationDB, CRAQ (Chain Replication with Apportioned Queries) for strong consistency, and a USRBIO userspace I/O API for bypassing kernel overhead. Likely uses RDMA verbs (InfiniBand) directly for data-plane transfers. FUSE interface appears supported for POSIX compatibility. Components likely include separate metadata, storage, and client layers based on typical designs of this class.
CI build badge present via GitHub Actions (build.yml). Test frameworks (gtest, gmock) are listed as build dependencies, suggesting unit tests exist. Coverage extent is not documented in README.
Last push was May 7, 2026, approximately 6.5 weeks before the evaluation date. This indicates active, ongoing maintenance rather than a one-time dump. The repository has received updates well past the initial February 2025 release burst, suggesting the internal team continues to develop it.
ADOPT IF: you are building or operating a large-scale GPU training cluster with InfiniBand/RDMA networking and NVMe SSDs, and need a storage layer tuned specifically for AI workloads including KV-cache offload, parallel checkpointing, and high-IOPS dataloader access. AVOID IF: you lack RDMA infrastructure, have a small to medium cluster, need mature multi-vendor support, or require a large existing operator community for production support — the operational complexity and hardware requirements are substantial. MONITOR IF: you are planning future large-scale AI infrastructure and want to track whether a broader operator ecosystem forms around 3FS over the next 12–18 months.
Independent dimensions
Mainstream potential
3/10
Technical importance
9/10
Adoption evidence
4/10
- Operational complexity is high: requires RDMA-capable networking (InfiniBand), FoundationDB as a dependency, and deep Linux systems expertise — significantly raising the barrier to adoption outside well-resourced AI labs.
- External community adoption is unverified; the project may remain primarily a reference implementation of DeepSeek's internal system rather than evolving to serve diverse operator needs.
- Hardware dependency on specific NIC vendors and NVMe configurations means deployment environments must be carefully matched to the architecture's assumptions — cloud or commodity setups may see much lower performance.
- Maintenance risk is concentrated: the project appears driven by a single organization. If DeepSeek's priorities shift, external contributors may be insufficient to sustain development.
- Documentation and operational guides appear limited relative to the system's complexity, which may impede adoption even among technically capable teams.
3FS is likely to remain a specialized reference system used by a small number of large AI labs and cloud providers capable of operating RDMA clusters, rather than achieving broad mainstream adoption. It may gain traction as a design reference for next-generation AI storage systems.
Newsletter
Get analyses like this every Monday
Free weekly digest of the most interesting open-source discoveries.
Languages
Information
- Language
- C++
- License
- MIT
- Last updated
- 2mo ago
- Created
- 17mo ago
- Analyzed with
- anthropic/claude-haiku-4-5
Stars over time
Contributors over time
Top 100 contributors only — repos with more will plateau at 100.
Open issues
why one service need multi config files?
deployment crashed at data placement part need expert to look into
cmake error after follow the readme guide
any plan to support erasure coding?
ARM架构平台编译通过，启动监控服务，folly::symbolizer::getStackTraceStr() 在aarch64架构上导致段错误，怎么解决？
Top contributors
Recent releases
No releases published yet.
Similar repos
seaweedfs/seaweedfs
SeaweedFS is a distributed object storage system designed to efficiently handle...
| Repository | Stars | Week Δ | Language | Score | Updated |
|---|---|---|---|---|---|
|
deepseek-ai/3FS
this
|
10k | +23 | C++ | 8/10 | 2mo ago |
 NVIDIA/aistore
|
1.9k | — | Go | 8/10 | 12h ago |
 happyfish100/fastdfs
|
9.2k | — | C | 7/10 | 1d ago |
 cubefs/cubefs
|
5.6k | — | Go | 8/10 | 1w ago |
|
seaweedfs/seaweedfs
|
33.4k | — | Go | 8/10 | 8h ago |
 facebookresearch/faiss
|
40.5k | — | C++ | 8/10 | 14h ago |
Also targets cloud-native and AI workloads, written in Go with broader community governance under CNCF. Less RDMA-focused and likely lower raw throughput ceiling, but has more production deployments documented across multiple organizations and better ecosystem integrations.
General-purpose distributed object/file storage with very large community. Not specifically optimized for AI training I/O patterns or RDMA; would require additional layers to match 3FS's KV-cache and checkpointing use cases.
The incumbent HPC distributed filesystem widely used in supercomputing and AI clusters. Battle-tested at massive scale but complex to operate. 3FS targets a similar performance tier but with simpler deployment targeting GPU clusters specifically.
Intel's open-source storage system also targeting RDMA/NVMe HPC workloads. Technically comparable in design philosophy but has longer history and broader vendor support. 3FS appears more narrowly tuned to deep learning pipeline patterns.
Lightweight and widely deployed in Asia for object/file storage, but designed for web-scale blob storage rather than high-throughput sequential/random I/O for training. Not a direct competitor in the AI infra tier where 3FS operates.
