FLAME is basically the dream for running AI agents that need system command access such as coding agents. However, I have concerns about scaling to a very high number of nodes (many thousands). I can just write application Elixir code and scalability is batteries included; incredible devx.
With distributed erlang, the default is for the nodes to not run in “hidden” mode and to instead create many-to-many connections to all nodes, which severely limits the overhead of each additional node that is added to the cluster. I noticed this seems to also extend to the default in FLAME as I did not see an option to support hidden node detection. (I may be wrong, was a fairly quick scan).
Would the solution be, in the case of the Fly backend, to run each node using a separate docker image that passes the “-hidden” flag then fork/update FLAME to support usage of nodes(hidden) for spawned workers? (Or maybe this is already achieved somehow). My ideal is the main cluster that runs the web servers and Oban workers for each agent is in the “non hidden” cluster, and they each only know about some number of dedicated node connections running each agent (a hub and spoke architecture).
Regardless, am curious to see what you think. Cheers.
