seaweedfs
by
chrislusf

Description: SeaweedFS is a fast distributed storage system for blobs, objects, files, and data lake, for billions of files! Blob store has O(1) disk seek, cloud tiering. Filer supports Cloud Drive, xDC replication, Kubernetes, POSIX FUSE mount, S3 API, S3 Gateway, Hadoop, WebDAV, encryption, Erasure Coding. Enterprise version is at seaweedfs.com.

View chrislusf/seaweedfs on GitHub ↗

Summary Information

Updated 1 hour ago
Added to GitGenius on March 2nd, 2026
Created on January 5th, 2026
Open Issues/Pull Requests: 0 (+0)
Number of forks: 2
Total Stargazers: 17 (+0)
Total Subscribers: 0 (+0)
Detailed Description

SeaweedFS is a fast, distributed storage system designed to handle massive amounts of data, specifically blobs, objects, files, and data lakes. Its primary purpose is to provide a scalable and efficient storage solution for billions of files, offering high performance and cost-effectiveness. The project is open-source, Apache-licensed, and actively developed with community support.

At its core, SeaweedFS functions as a blob store, optimized for handling small files with exceptional speed. It achieves this through an architecture that minimizes the load on a central master server. Instead of managing all file metadata centrally, the master primarily oversees volumes residing on volume servers. These volume servers, in turn, manage the files and their metadata. This distributed approach alleviates concurrency bottlenecks and distributes metadata across the system, leading to faster file access times, often requiring only a single disk read operation (O(1) disk seek).

Key features of the blob store include its O(1) disk seek performance, cloud tiering capabilities, and support for various replication levels, including rack and data center awareness. It offers automatic master server failover, eliminating single points of failure. Data compression based on file MIME type, and automatic compaction to reclaim disk space after deletions or updates are also supported. Furthermore, SeaweedFS provides automatic entry TTL (Time-To-Live) expiration, flexible capacity expansion, and the ability to add or remove servers without requiring data rebalancing. Customizable storage tiers allow users to balance performance and cost, and transparent cloud integration enables unlimited capacity through tiered cloud storage for warm data. Erasure coding is implemented for warm storage, reducing storage costs and increasing availability.

Beyond the blob store, SeaweedFS offers a Filer component, which adds directory and POSIX attribute support. The Filer is a linearly scalable, stateless server that can integrate with various metadata stores like MySQL, PostgreSQL, Redis, and others. The Filer provides features such as file TTL for automatic metadata and data expiration, FUSE mount for direct file access as a local directory, and active-active replication for cross-cluster data synchronization. It also supports an Amazon S3 compatible API, Hadoop integration, asynchronous replication to the cloud for backups, WebDAV access, AES256-GCM encrypted storage, and the ability to handle super large files. Cloud Drive functionality allows for mounting cloud storage to the local cluster, caching data for fast read/write operations with asynchronous write-back. Additionally, it offers a gateway to remote object storage, mirroring bucket operations.

SeaweedFS also integrates with Kubernetes through a CSI (Container Storage Interface) driver and an operator, simplifying deployment and management in containerized environments.

The project provides comprehensive documentation, including a wiki, white papers, and introduction slides. It offers quick start guides for various deployment scenarios, including a "weed mini" setup for development and testing, Docker-based S3 API access, and single-binary deployments. The quick start guides demonstrate the ease of setting up and using SeaweedFS.

In essence, SeaweedFS is a versatile storage solution designed for scalability, performance, and cost-effectiveness. Its architecture, features, and integrations make it suitable for a wide range of use cases, from handling billions of files to integrating with cloud storage and containerized environments. The project's open-source nature and active community support further contribute to its appeal and ongoing development.

seaweedfs
by
chrislusfchrislusf/seaweedfs

Repository Details

Fetching additional details & charts...