I was mostly suggesting that non-distributed local-node caching is simple to do with Erlang. If you don’t need to distribute the state, then running a background job processing (which was the original problem of this thread) is as easy as starting a process from the request handler ![]()
My feeling is that people go to Redis because they used it before, everyone else uses it, and it’s seductively simple. Many, though, seem to disregard the fact that this thing has to be set up and configured somewhere, and it’s either a single point of failure (non-distributed), or otherwise unreliable.
A non-distributed in-memory cache can be as easy as ETS table, or even an Agent for smaller throughputs. Making it persistent can be easily done with DETS or non-distributed mnesia.
When it comes to managing a distributed state, the proper solution is not going to be simple with Erlang, but I don’t think distributed is ever simple or easy. Minimizing a cluster-wide state would be the first thing I’d consider. Otherwise, I’d look at libraries such as Phoenix Presence, riak_core, riak_ensemble, swarm, or syn, depending on the particular case.
Of course, reaching for 3rd party external components and databases is always a reasonable option, especially since using them can be simpler than evaluating Elixir/Erlang libraries and setting everything up properly in the code. Having a database as the single source of truth will work, but it will also be a single point of failure and a possible bottleneck. If you want to scale it, the db has to become distributed, which will then lead to similar challenges.


















