So before I suggest anything else, I will say one last time that you should at least try a fully centralized approach first, where you store all topic metadata in one ets table
Yes I totally agree and this is what I’ve done done so far.
I basically store all the counts in 1 ETS table and I have a balancer algorithm running with highlander (only executed in a single node across the whole cluster doing the balancer for all rooms)
What I am suggesting here is that you turn this consistency problem inside-out. Instead of trying to store an eventually consistent “user count” of each room on every node, you should store some room counts, un-replicated, on each node. So each room has a node that it “lives” on, which maintains its count. This is called partitioning, or sharding.
Sharding sounds quite nice yes and I thought about it… BUT the only issue is that if I don’t store all the room counts in the same ETS but partitionned them across nodes, that means my balancer which re-equilibrates rooms won’t be able to do it “globally”, but only with the rooms in its nodes, so there might some room left out…
For example:
Node 1, you could have :
room-1 => 30 users
room-2 => 40 users
Node 2, you could have :
room-3 => 15 users
room-5 => 20 users
Node 3, you could have :
room-4 => 5 users left
Since rooms are partitioned, my balancer won’t be able to merge room-4 with rooms 1, 3 and 5.
Instead
balancer one node 1 will merge 1 and 2 together
balancer one node 2 will merge 3 and 5 together
Room 4 cannot be merged with anything.
