I don’ t think wrapping Python with Elixir is a very good Idea. Python is slow and already a glue language so that’s like 2 glue layers. For performance you should interface with a faster lower level language.
“there is quite a demand to “ease” the actual big-data pipe.”
I’m not sure this is true. Have you even studied Spark? You can program it already in 4 languages: Scala (the native language it’s programmed in. This is actually a high level language but fast (& also complex unfortunately)), Java, R & Python. So for the Pythonistas: there really is no need to recreate Spark in Python since it already has a Python interface. Next to that you do do a lot of data retrieval/manipulation in SQL (for non-prgrammers) & there are interactive notebooks (like in iPython).
Nothing prevents you in principle to develop an Elixir interface to Spark but to make this workable you would need efficient (mutable) datatstructures to represent dataframes & large arrays (vectors/matrices/tensors) in Elixir and this does not exist at the time. And no a (nested) (linked) list is NOT going to cut it for representing vectors/matrices/tensors in Elixir.
