I would suggest actually taking the models you are wanting to use for a spin using Livebook.
Right now today you can import models developed in say python and operationalize them using Elxir with Bumblebee, Ortex, axon_onnx, Axon, Scholar etc all underpinned by NX.serving which can provide distributed serving of models using every GPU in your cluster.
You can’t make robust decisions from the armchair and will need to do some validation for yourself. Whilst Elixir is realitively new to the ML space it has the underpinnings to be the compelling solution for deployment. The weakest area currently is model development, but for serving models Elixir has tools to import existing models. That’s not to say there are not gaps but the important thing to identify is are there untenable gaps for the use cases you currently have and if you do use Elixir then what parts of your current enviornment will it replace. I would hazard to guess you would pick a scoped part of the overall solution and try Elixir there, get some experience then consolidate and expand from that.
One of the most advanced intellectual property search systems migrated all their models and processing to Elixir to operationalize their solution. In doing so they halved their AWS costs through less complexity processing 100’s of millions of patents on a weekly basis when updating their models. Their original talk is here:
Their latest talk is here:
