Pub/sub and fan-out for live chat
Updated Nov 10, 2026 · 3 min read
Live chat looks simple. Someone types, everyone sees it. The hard part is the “everyone”. One message in a room of 200,000 viewers is 200,000 deliveries, spread over dozens of servers, within a second or two. Pub/sub is how those servers share messages without knowing about each other, and fan-out is the cost you have to plan for.
WebSocket gateways
Each viewer keeps one WebSocket open to a gateway server, so the server can push messages the moment they exist instead of waiting to be asked. A gateway mostly holds connections and copies bytes, so one machine can hold tens of thousands of sockets. A big room needs many gateways, and its viewers end up spread across all of them.
Publish once, deliver to subscribers
When a message arrives at one gateway, the others need it too. Rather than every gateway talking to every other, each room gets a channel on a pub/sub broker such as Redis, NATS or Kafka. A gateway subscribes to a room’s channel while it has viewers in that room, and unsubscribes when the last one leaves. The gateway that receives a message publishes it once, the broker hands one copy to each subscribed gateway, and each gateway pushes it to its own viewers.
Nobody has to keep a list of which gateway serves whom. Gateways come and go just by subscribing and unsubscribing, and the broker only ever sends one copy of a message to each gateway, however many viewers sit behind it.
The fan-out maths
Deliveries a second are messages a second times viewers. The trap is that messages grow with viewers too, because a bigger audience has more people chatting. If one in 200 viewers chats and each sends a message every 20 seconds, a room of 100,000 makes 25 messages a second and 2.5 million deliveries. Ten times the audience makes 250 messages a second and 250 million deliveries, a hundred times the work.
Keeping big rooms readable
Past a point, more servers is the wrong answer, because nobody can read hundreds of messages a second anyway. These are the tools platforms use, and they solve different problems.
| Technique | What it does | What it doesn’t do |
|---|---|---|
| Slow mode | Each chatter may send one message every few seconds | Cap the crowd. 60,000 chatters at one message a minute is still 1,000 a second |
| Per-user limits | Stops one bot or script flooding the room | Help when the flood is many real people |
| Batching | Sends a viewer’s new messages in one frame every few hundred ms | Cut the bytes, since every message still goes to everyone |
| Sampling | Sends each viewer a readable share, say 10 messages a second | Show everything, so keep the streamer’s, moderators’ and the viewer’s own |
Slow consumers
Some viewers sit on bad mobile connections and can’t keep up. If a gateway buffers everything for them, its memory grows until it falls over. Give each connection a small buffer, drop the oldest chat when it fills, since old chat is worthless, and disconnect anyone who stays behind. The app reconnects and carries on from the latest messages.
What chat can get away with
Live chat can lose a message now and then, and nobody notices. That lets the delivery path stay fast and simple, with no acknowledgements per viewer. Messages are still saved, in order, for replays and moderation, but that write happens beside the live path rather than in front of it. A one-to-one chat app makes the opposite choice, because there every message matters.