← All articles

A design review checklist for the flaws that hurt first

Updated Nov 10, 2026 · 4 min read

A design review is cheap insurance. A diagram and a page of text can hide a flaw that would take a week of incidents to find in production. The same few flaws turn up again and again, and each one has a question that exposes it. Ask what happens when a part dies, when a request repeats, when data goes stale, and when two requests race.

One of anything

Walk the diagram and count. Any box with no second copy is a single point of failure, and the dangerous ones are usually small and easy to overlook, like one Redis node holding sessions, one cron server, or one DNS provider. Managed services that already run across zones are fine. A database with a standby that takes over on its own is a short blip. A node with no copy at all is an outage.

Requests that repeat

Networks drop answers. A client that times out doesn’t know whether its request failed or only its reply did, so it retries. For reads that’s harmless. For anything that moves money or creates something, the retry repeats the action. The fix is an idempotency key. The client makes one random key per action and sends it with every try, the server stores the result under that key, and a repeat gets the first result back.

Retries that multiply

When services call each other in a chain and each one retries, the retries multiply. If each of 3 services tries 4 times, one user request can become 4 × 4 × 4 = 64 attempts on the database, right when it is struggling to recover.

1 layer4
2 layers16
3 layers64
4 layers256
Attempts on the database for one request when it is down, with 3 retries at each layer. This uses a log scale, so equal steps in length mean multiplying, not adding, so bars that look close can be 1,000× apart.
  • Retry in one layer only, usually the one closest to the user, and let the others fail fast.
  • Wait longer after each failure, and add randomness so clients don’t retry in lockstep.
  • Cap retries with a budget, for example no more than 10% extra load.
  • Set a timeout on every call, so a hung dependency can’t hold your threads forever.

Caches in front of money

A cache serves a copy, and copies go stale. That is fine for a product page, where a price an hour old is a small problem. It is not fine at checkout, where it means charging a price that no longer exists. Keep caches for display, and read anything you charge for from the source of truth, or check that what the user saw is still current.

Keys everyone writes to

Sharding spreads load only if the key spreads it. A key that everyone writes at the same moment, like today’s date or a single popular account, sends all writes to one shard while the rest sit idle. That’s a hot shard. Pick a key with many values in use at once, such as a hash of the user or device id, and check how your busiest moment spreads, not your average one.

Check, then act

Code that reads a value, decides, then writes leaves a gap where another request slips in. A single-use coupon gets used twice, the last seat gets sold twice. Do the check and the change in one statement, such as an update that only matches when the coupon is unused, and act only if it changed a row. A unique constraint does the same job for inserts.

Two writes that should be one

Committing to a database and then publishing an event are two separate steps, and a crash between them leaves the database and the rest of the world disagreeing. A transactional outbox fixes it. Write the event to an outbox table in the same transaction as the change, and let a separate relay publish it. The relay may publish twice, so consumers ignore repeats.

Waiting on someone else

Anything a request waits for adds its slowness and its outages to yours. If checkout waits for the email provider, checkout is down when the email provider is. Ask of every call whether the user needs its answer right now. If not, put it on a queue and do it after you reply.

The checklist

AskWhat goes wrongUsual fix
Is there one of anything?It dies and takes everything with itA second copy that takes over on its own
What if this request repeats?Money moves twiceIdempotency key
How do retries add up?A blip becomes a stormRetry in one layer, with backoff, jitter and a budget
Can a cached value be charged?Stale prices on real paymentsRead from the source of truth at checkout
Where does the busiest moment land?One hot shardA key with many values in use at once
Can two requests race here?Used twice, sold twiceOne conditional statement, or a unique constraint
Are two systems written separately?They disagree after a crashTransactional outbox
Does the user need this answer now?Someone else’s outage is yoursA queue and a timeout