Hi all, I’m new to Elixir and not yet super experienced in distributed systems in general and just recently started learning. I’m pretty interested and really like what I’ve seen of Elixir so far!
This might be a stupid question, and apologies in advance if it is the wrong place to ask: I absolutely love the premise of Elixir (reducing a lot of the complexity most of our industry just takes for granted) and have read people online say it basically renders things such as Kafka as unnecessary. I don’t have any experience with Kafka itself, but I have previously worked with Akka, and read enough to get the feeling that in that community, distributed actors and Kafka are seen more as apples and oranges. So I guess I’m wondering how is it different with Elixir? I know Akka is heavily inspired by Erlang, but don’t know Elixir and Erlang well enough to fully know where the similarities end and the differences start.
The very basic stuff I’ve seen so far: it seems Elixir/Erlang makes distributed processes communicating seamlessly from different nodes work more out of the box, while with Akka it does take more configuring (at least IME, back when I worked with it in 2019, some config knowhow and reading the docs to get that working). There’s also GC working differently where in the BEAM it takes place per process, while with Akka, due to it working on top of the JVM, it occurs more globally at the level of the entire node’s JVM (or at least it used to be this way, not sure about newer and/or specialty versions of the JVM).
I could be mistaken, but I get the feeling a reason you’d probably still want to use Kafka even with Akka is, if besides using actors communicating across nodes on a cluster and fault tolerance, you also want to provide stronger guarantees that your message is going to get delivered by taking advantage of Kafka being a persistent log (I can see this being useful for a financial transaction, where the user having to try again could potentially be stressful because they will have to wonder if they’ll get charged twice or something).
So in a nutshell: how would one eliminate the need for this external piece of infrastructure by using Elixir?
