There are a few more steps usually involved that wrap around LLM, either on the indexing, retrieval or formulation of the responses phase and I am not sure how much of these tools we have in our ecosystem. If you look at LlamaIndex, for example, you can pick from several strategies for querying/retrieval, pre-processing and post-processing of data, summarization steps, verification of alignment steps, context/window tracking, logging and such. They work, out of the box, for the most part.
I think it’s feasible to build an RAG tool in plain Elixir but you have to be prepared to build more of these building blocks yourself.
