There are far more rooms than nodes, so I don’t think it would be too big a deal. But if you have enough nodes that you start to get some skew from the distribution, you can use the same load balancing algorithm across nodes. Pick two nodes and route the join to the emptier one.
You can keep an eventually consistent “load” for each other node on each node. There are only a handful of nodes so this is much easier to keep in sync by gossiping, and having exact information matters much less because the load on a particular server is not going to change by that much in a few seconds.
A simple way would be to destroy room 4 and send all of its users through the routing algorithm separately. But if you want it to work as you say, there is no reason you can’t add users to rooms “as a batch”. It’s just more logic in the load balancer.
The logic here is not trivial though. For example, do you want to merge room 4 off of that node? Or do you want to take some users from the other rooms and send them to room 4? After all, if you remove those users from node 3, it will be empty, which is not very balanced!
In practice, since there are so many more rooms than nodes, I don’t think you would run into this problem. There should be enough rooms on each node that you can balance them entirely intra-node. If not, get rid of some nodes! 
