1.5 How to Implement Distributed Storage

1. What Is Distributed Storage?

File storage in a distributed environment (across multiple machines).

2. Why Is Distributed Storage Needed?

A single machine has limited performance and capacity.

A single machine does not have high availability.

3. How to Implement Distributed Storage

• Distributed-System Replication
• Distributed-System Partitioning
• Distributed Consistency
• Distributed-System Cluster Metadata Management

4. Distributed-Storage Examples

4.1. Elasticsearch

4.2. Redis

4.3. MySQL

4.4. FastDFS

FastDFS is a distributed file system. As the name suggests, since it is a file system, it is of course used to store files and provide upload and download services externally. Traditional FTP consists of only one server, which is far from sufficient for the massive amount of Internet data. FastDFS provides clustering capabilities and includes features such as load balancing and expansion.

4.4.1. Architecture Analysis

The FastDFS architecture mainly consists of two parts: Tracker server and Storage server.

The specific working mode is as follows. When uploading a file, the client first sends a request to the file server.

This request is received by the Tracker server, which selects an appropriate Storage server through a series of scheduling algorithms and then returns the address of the Storage server to the client.

The client uploads the file to this Storage server.

Downloading is similar. The client sends a request to the Tracker server to download a file. The Tracker server checks whether the file exists and which Storage server specifically stores it, then returns the address to the client. The client then downloads the file from that address.

5. References

• happyfish100/fastdfs: FastDFS is an open source high performance distributed file system (DFS). It’s major functions include: file storing, file syncing and file accessing, and design for high capacity and load balance. Wechat/Weixin public account (Chinese Language): fastdfs
• FastDFS + Nginx module - morrowind - 51CTO Blog