Thanks for replying!! Yea no worries, it’s a bit of code, thanks for looking it over! Sorry if I am misunderstanding your suggestions in my reply below!
it seems like this is something you might be able to handle with core library functions before reaching for GenServer,
Enum.chunk_everyandTask.async_streammaybe
Yea I was wondering about this! I don’t think so unfortunately, because while Enum.chunk_every would help me split up a list of items into chunks, it wouldn’t help me batch a sequence of individual calls. I want to make a bunch of individual calls like so:
out1 = api(in1)
out2 = api(in2)
...
outN = api(inN)
and behind the scenes batch them into one API call. I actually already use chunk_every and Task.async_stream in my manual batching implementation, but it’s annoying because I have to keep track of keys for each input. In particular I can’t just write
inputs
|> Enum.map(fn input -> Task.async(fn -> api(input) end) end)
|> Task.await_many()
… anymore, instead I have to write something like
inputs
|> fn input -> {key_for_input(input), input} end
|> Enum.chunk_every(50)
|> Task.async_stream (fn batch
# the api only cares about the values...
api(Enum.map(batch, &(elem(&1, 1)))
# but we need to reassociate the output values with their keys afterwards
|> Enum.zip(Enum.map(batch, &(elem(&1, 0)))
)
|> Enum.to_list
|> Enum.flat_map(...)
And I have to write this for every new API call…
I think I can’t use handle_cast because I do care about the results, e.g. I want to still be able to write code like this:
out1 = api(in1)
out2 = api(in2)
...
outN = api(inN)
I just want it to be batched behind the scenes.
I saw this library for Erlang: GitHub - rabbitmq/gen-batch-server: A generic batching server for Erlang and Elixir · GitHub but I tried to understand what was going on and it seems very lightly documented.
