Elixir in an environment of polyglot microservices; feasible or not?

Questions & Help Questions
, ,
1

My question: considering an application consisting of a set of polyglot [micro ~ substitute your definition]-services, would Elixir, and in extension Erlang, fit into such an environment?

Disclaimer: I am new to Elixir. I’ve started learning and experimenting only a few weeks ago, and do not mean to offend with an unthoughtful question, I just didn’t come across satisfying answers to this problem yet. :drooling_face:

There have been several questions to this avail, and from the discussion around those, as well as from two excellent articles - [Dawn of the Microlith](http://Dawn of the Microlith), http://blog.plataformatec.com.br/2015/06/elixir-in-times-of-microservices/ - I have gathered that the unique features of Erlang and Elixir, namely the BEAM and Elixir Umbrella-Projects, much of the real world benefits of a micro service-oriented architecture can be had in Elixir, without some of the complex drawbacks of such a system - all while not actually writing micro-services.

What is still an open question to me, however, is how BEAM-bound languages, Elixir in particular, can cope in an environment in which they’re not the sole, or even a privileged, resident - namely as part of a set of polyglot micro services, written in different languages and using vastly different tech-stacks.

While I’ve heard that the BEAM has performance related issues when communicating with external processes, I haven’t found much concrete information of Elixir or Erlang in such a context, nor am I even aware of the preferred, if there’s such a thing, way of handling inter-process communication with the BEAM.
While I’ve read up briefly on ports and such, I haven’t gained much clarity yet.

So, I guess to sum it up, there is a core question in my mind:
Is Elixir fit to run, as a service and containerized, besides a myriad of other, distributed, not-necessarily-Elixir services?

2

Just some examples:

It’s is sometimes suggested that Erlang/Elixir can be used with the BEAM to choreograph activities. However a lot of things can be done quite effectively on the BEAM itself so interactions are typically restricted to scenarios where there is a significant, tangible benefit to doing something in a different environment.

In terms of containerization it becomes a question of what role Erlang/Elixir has to play in terms of controlling distribution as your container management system is largely calling the shots. The BEAM was designed before containers arrived on the scene and doesn’t necessarily benefit from the extra layer of indirection that containers impose. Erlang and the BEAM have a bit of a different view of what constitutes reliability and high availability - while processes come and go if designed properly, the “system” should rarely need to be down (i.e. cycled or re-cycled) even in the face of single node failures (hardware or software).

I would expect that a system designed around OTP/BEAM would consume much fewer resources compared to an on-demand container-based management system that is more targeted toward the elastic consumption of readily available commodity server resources.

https://medium.com/@pentacent/getting-started-with-elixir-docker-982e2a16213c

https://blog.ispirata.com/clustering-elixir-erlang-applications-in-kubernetes-part-1-the-theory-ca658acbf101

https://medium.com/polyscribe/a-complete-guide-to-deploying-elixir-phoenix-applications-on-kubernetes-part-1-setting-up-d88b35b64dcd

https://medium.com/polyscribe/a-complete-guide-to-deploying-elixir-phoenix-applications-on-kubernetes-part-2-docker-and-81e934c3fceb

https://medium.com/polyscribe/a-complete-guide-to-deploying-elixir-phoenix-applications-on-kubernetes-part-3-deploying-to-bd5b1fcbef87

https://medium.com/polyscribe/a-complete-guide-to-deploying-elixir-phoenix-applications-on-kubernetes-part-4-secret-f851d575bdd1

If I recall correctly Dawn of the Microlith was more about avoidance of the overhead of micro services by sticking to systems designed around the BEAM. In a sense, instead of building a microservice around a bounded context you would build an OTP application (or a set of them) instead.

3

As long as you’re using open standards for communication between services. It generally doesn’t matter what language you choose. Erlang and Elixir speak most of those open protocols like HTTP, AMQP, etc. I’d say, you’d be hard pressed to find a language that couldn’t fit.

4

The polyglot use case would be a viable way to introduce elixir to a bigger audience.

I have a java http server that calls Python, c# grpc workers to build a response. I thought about using elixir/Phoenix as a the communication layer dropping the need of grpc. However I am still confused about how to go about this I am unsure if the Approach should be inside beam or using a more agnostic solution PubSub with Redis backend. Ideally I would like queue the workload and consume it as a limited amount of workers finish their take (to avoid chocking the server)

PubSub with Redis sounds like a better agnostic and stablished solution to me but if you can make a case for a was to do it inside beam I will listen. Ideally the extra glue code should be short and wel maintained (and support as much Langs as possible)

I would love to see an example where we create polyglot workers that sleep and then return a string to the caller. Please refer to me if one exists otherwise I would love a little advise on what would be the right way to go about this.

I know I am almost reducing Phoenix to a generic pub sub at this point but I think that would be a great way to introduce traditional companies that are in microlith hell. First you glue your services with beam and then you start coding more in elixir.