NOTE
3.4 Redis Blocking
What Redis blocking is, how to discover it, and internal and external causes including complex commands, CPU saturation, persistence, swap, and network problems.
This is a historical learning note and may contain outdated or incomplete understanding.
1. What Is Redis Blocking?
- Redis takes a long time to respond to client requests.
2. Why Blocking Happens
In essence, Redis has a single-threaded architecture and command execution is completed in this thread.
3. How to Discover Blocking
- A business-side Redis client times out when connecting to the Redis server, triggering a monitoring alert.
4. Cause Analysis
4.1. Internal
4.1.1. High-Complexity Commands Are Used
- Unreasonable data structures
- How to discover:
- Check the slow-query log: Redis Slow Queries
- How to solve:
- How to discover:
- Too much data in an atomic command, for example
msetwith 10K pieces of data.- How to solve: execute in batches; if atomicity is not required, pipeline can be used.
4.1.2. CPU Saturation (Single-Machine Performance Bottleneck)
- CPU saturation means Redis drives a single CPU core close to 100% utilization.
- How to discover:
- Use the
topcommand to easily identify the CPU utilization of the corresponding Redis process.
- Use the
- How to solve:
- In essence, this is a single-instance Redis performance bottleneck, so Redis Cluster mode is needed to distribute the pressure.
4.1.3. Persistence
- Some persistence operations can block the main thread: fork blocking and AOF flush blocking.
- Fork blocking
- RDB and AOF rewriting fork a child process for processing.
- How to discover: use
info statsand checklatest_fork_usec. - How to solve: avoid excessively large memory instances.
- AOF flush blocking
4.2. External
4.2.1. CPU Competition
For example, Redis is deployed together with other services on the same machine.
4.2.2. Memory Swap
Set /proc/sys/vm/swapness to 1, retaining swap while reducing its use.
4.2.3. Network Problems
4.2.3.1. Connection Refusal
- Network interruption: for example, network switching or bandwidth exhaustion;
sarcan be used for diagnosis. - Redis maximum number of client connections: Redis configuration file.
- Connection overflow: for example, the Linux process file-descriptor limit.
4.2.3.2. Network Latency
Depends on the network environment from client -> server.
Discussion
Sign in with GitHub to comment. Discussions are stored as GitHub Issues.View on GitHub