FLAME is basically the dream for running AI agents that need system command access such as coding agents. However, I have concerns about scaling to a very high number of nodes (many thousands). I can just write application Elixir code and scalability is batteries included; incredible devx.
With distributed erlang, the default is for the nodes to not run in “hidden” mode and to instead create many-to-many connections to all nodes, which severely limits the overhead of each additional node that is added to the cluster. I noticed this seems to also extend to the default in FLAME as I did not see an option to support hidden node detection. (I may be wrong, was a fairly quick scan).
Would the solution be, in the case of the Fly backend, to run each node using a separate docker image that passes the “-hidden” flag then fork/update FLAME to support usage of nodes(hidden) for spawned workers? (Or maybe this is already achieved somehow). My ideal is the main cluster that runs the web servers and Oban workers for each agent is in the “non hidden” cluster, and they each only know about some number of dedicated node connections running each agent (a hub and spoke architecture).
Regardless, am curious to see what you think. Cheers.
Maybe it’s worth considerding libcluster with partisan and a custom partitioning mechanism. this way you could avoid the mesh & decide how to split your nodes into small clusters => you could run flame in each small cluster
Hijacking with a noob question here: partisan solves a different problem than pg’s process group scopes, correct?
I assume partisan avoids the full mesh at the network level and process group scopes create an overlay network within the full mesh used for group membership propagation?
it’s not a problem of pg but actually a problem of how disterl works… it is explained on partisan docs
Erlang/OTP, specifically distributed erlang (a.k.a. disterl), uses a full-mesh overlay network. This means that in the worst case scenario all nodes are connected-to and communicate-with all other nodes in the system. Failure detector. These nodes send periodic heartbeat messages to their connected nodes and deem a node “failed” or “unreachable” when it misses a certain number of heartbeat messages i.e. the net_tick_time setting in disterl.
Thank you - I did and that lead me to the question of if they solve two fundamentally different problems (specifically as described), not if partisan solves a problem with pg.
note that FLAME nodes are assumed trusted nodes (shared erlang cookie, network access to everyone, mutable beamfiles on disk, etc), so you cannot execute untrusted code there unless you are running that execution in some sandbox env of its own.
I was thinking of running them as a highly limited linux user in the containers with only access to their own git repos as well as cli dev tools. It’s a convenient way to continue to leverage distributed pubsub and message passing per agent. LMK if there’s any possibilities of vulnerabilities there!
Are you aware if it’s possible to connections from FLAME nodes to be exclusive to the parent?
