Scaling Real-Time Apps Is About Rooms, Not Connections

Every capacity conversation about real-time apps starts with the wrong number. People ask how many connections a server can hold, as if that were the constraint. It rarely is. The thing that actually determines whether your real-time app holds up is fan-out: how many clients each event has to be delivered to, multiplied by how often events fire, across how many rooms. A hundred thousand idle connections are cheap. A thousand connections in one room where everyone is generating events for everyone else can be brutal. If you want to reason about real-time scale correctly, count messages delivered, not sockets open.

Connections are cheap; delivery is not

An open WebSocket that is doing nothing costs very little. It is a bit of memory and a file descriptor. Modern servers hold enormous numbers of idle connections without breaking a sweat, and every benchmark that brags about connection counts is measuring idle sockets. That number tells you almost nothing about whether your app will survive real usage.

The work happens when an event fires and has to reach subscribers. Send one message into a room of N people and the server does N deliveries. If those N people are each generating events at some rate, the total delivery work is roughly the number of people times the number of people times the event rate. That squares. A room of 10 is trivial. A room of 1,000 where everyone is active is a million deliveries per event cycle. The connection count barely moved; the delivery work exploded. This quadratic blowup is the trap I flagged in real-time backend mistakes at scale, and it is invisible until you have a busy room.

Room shape is the real design variable

Because cost is driven by fan-out, the shape of your rooms is the lever that matters. Many small rooms scale beautifully: a chat app with thousands of two-person and small-group conversations spreads its fan-out thin, and total load grows gently. A few enormous rooms are where things get hard: a live event with a hundred thousand viewers all in one channel is a genuinely different engineering problem than a hundred thousand people spread across small rooms, even though the connection count is identical.

So the first scaling question is not "how many connections" but "what does my biggest room look like, and how active is it." If your product naturally shards into small rooms, you have an easy problem and you should not over-engineer. If your product has a few giant rooms, you need specific strategies for those rooms, and your channel design has to reflect it. This is downstream of pub/sub channel granularity: the channel is the room, and the room is where fan-out lives.

Strategies for the big rooms

When you genuinely have large, active rooms, a few techniques keep them alive. Coalesce updates so a client that is behind gets the current state instead of every intermediate step, which cuts delivery volume dramatically for anything that is really just changing state. Throttle broadcast frequency so a room updating a thousand times a second sends clients a consolidated view a few times a second instead. For read-heavy broadcast rooms, treat it as one-to-many distribution rather than many-to-many chat, because the fan-out pattern is completely different.

And spread the room across servers with a backplane so no single machine has to do the entire fan-out for a huge room. That is a horizontal scaling concern I get into in scaling WebSockets: a shared pub/sub layer lets multiple socket servers each handle a slice of a big room's members while still delivering every event to everyone. Without that, one giant room is capped by one machine.

Count the right thing, then decide build or buy

The practical discipline is to model your load as deliveries, not connections. Take your busiest realistic room, multiply members by event rate by members, and that is your real number. Do that before you pick infrastructure and you will make far better decisions than anyone benchmarking idle sockets.

This is also where the build-versus-buy question gets sharper. Handling small rooms is easy enough that a naive server works. Handling large, active rooms, with coalescing, throttling, and a cross-server backplane, is real distributed-systems work that most teams should not build from scratch. I would rather run that on a platform designed for it and spend my time on the product, which is why I use AltoHost for the real-time layer. If you are weighing that tradeoff, I laid out the whole decision in build vs buy real-time infrastructure.

Stop asking how many connections you can hold. Ask how many messages you have to deliver, and where. The answer to that question is the one that predicts whether your app survives its best day, and it is the number you should design around from the start.