Caching and async queues keep the hot path fast
Updated Nov 8, 2026 · 3 min read
Most systems have a hot path, the request that happens far more often than any other. For a link shortener it is the redirect. Two tools keep that path fast. A cache answers repeat reads from memory so they never reach the database, and a queue lets the request hand off work it does not need to wait for.
Cache-aside, step by step
The most common pattern puts the application in charge of the cache:
- Look up the key in the cache (Redis, Memcached, or a map in process memory).
- On a hit, return the value. Done.
- On a miss, read from the database.
- Write the value into the cache with an expiry (TTL), then return it.
Popular items stay in the cache because they keep getting read; rarely used ones expire or get evicted, usually least-recently-used first. Memory is spent on what people actually ask for.
What a good hit rate buys
Say a cache read takes 0.5 ms, a database read 5 ms, and 95% of lookups hit the cache.
| No cache | 95% hit rate | |
|---|---|---|
| Average read time | 5 ms | 0.95 × 0.5 + 0.05 × 5.5 ≈ 0.75 ms |
| Database reads per 10,000 requests | 10,000 | 500 |
Reads get almost 7× faster, and the database sees 20× less traffic. The second number is often the bigger win, because it means a smaller database, fewer replicas, and much more headroom for traffic spikes.
Keeping cached data fresh
- Set a TTL on every entry. It bounds how stale anything can get, even if you forget to clear it somewhere.
- Delete the cache entry when the underlying row changes. Deleting is safer than updating in place, because the next read rebuilds it from the source of truth.
- Write-through caches update the cache and the database together on every write. Reads are always fresh, but writes pay twice.
- Data that never changes, like a short code’s destination when links are immutable, can be cached with a long TTL and almost no invalidation logic.
The stampede problem
When a very popular key expires, every request that misses at that moment goes to the database at once. A link shared in a big campaign can turn one expiry into thousands of identical queries. Three common defenses:
- Request coalescing lets one request rebuild the entry while the others wait for its result.
- Jittered TTLs add a random few percent to each expiry so keys loaded together do not all expire together.
- Refresh ahead rebuilds hot keys shortly before they expire, in the background.
Queues let you do it later, on purpose
Some work has to happen because of a request, but the person making it does not need to wait. Counting a click, sending an email and resizing an image are typical. Instead of doing that work inline, the request appends a small message to a queue (Kafka, SQS, RabbitMQ, Redis streams) and returns. Workers read from the queue and do the work at their own pace.
- Responses stay fast. Appending to a queue takes about a millisecond, whatever the work behind it costs.
- Spikes get smoothed out. A minute with 50,000 clicks becomes a backlog workers drain over the next few minutes, instead of 50,000 concurrent database writes.
- Work gets batched. A worker can turn 1,000 click events into one write that adds 1,000 to a counter.
- Failures stay isolated. If the analytics database is down, redirects keep working and events wait in the queue.
What queues cost
- Results show up seconds or minutes later. Watch the queue’s lag and alert when it grows.
- You can get duplicates. Most queues deliver at least once, so a message can be processed twice after a crash. Give events ids and make the consumer ignore ones it has already seen.
- It is one more system to run. The queue needs capacity, monitoring and a plan for when it fills up.
How the parts fit together
| Part | Job |
|---|---|
| Cache | Serves hot links from memory in about a millisecond |
| Database | The durable home of every link |
| Key generator | Hands out unique short codes ahead of time |
| Click queue | Buffers click events so they can be counted later |