Designing a Cache Middleware

1. What Is Cache Middleware?

• General-purpose cache infrastructure.

2. Why Do We Need Cache Middleware?

• It hides details such as cache reads/writes, concurrency safety, cache eviction, and distributed support from the application layer.

3. How to Design a Cache Component

3.1. Basic Reads and Writes

• For example, a HashMap can provide O(1) read and write efficiency.

3.2. Concurrency Safety

Problems occur when multiple threads read and write the cache at the same time. How do we solve them?

3.2.1. Locking

• To avoid coarse-grained locks reducing concurrency, use the idea of JDK 1.7 ConcurrentHashMap:
  • Divide the cache into multiple shards and give each shard a lock. If clients update different shards, they do not need to wait for each other.
• sync.map
• JDK 1.8 ConcurrentHashMap.md
• JDK 1.7 ConcurrentHashMap.md

3.2.2. Asynchronous Updates Through a Log

• Following the design of a database, write all updates to a log and let a background process read the log and update the cache.

3.3. Cache Eviction Policies

• Cache Replacement Policies

3.4. Distributed Cache

3.4.1. Replicated Cache

• In-process cache + Distributed-System Replication
• Advantages: efficient in-process access.
• Disadvantages:
  • Nodes need to synchronize data; the more nodes there are, the slower the synchronization.
  • Poor consistency.

3.4.2. Centralized Cache

• A separate cache process + network access.
• Advantages:
  • Nodes do not need to synchronize data.
  • Better consistency.
• Disadvantages:
  • Network access makes it somewhat less efficient.

4. Examples

4.1. Redis

• Redis

4.2. Caffeine

5. References

• Implementing an LRU Cache - Bilibili
• Design a Cache System