There are some tools to leverage high import, for example GenStage, which will provide back pressure support.
I would also reach for insert_all, because it’s more efficient to send one query with 10’000 insert than 10’000 single insert.
Enum.chunk_by can be useful to treat smaller pieces.
But all of this does not fit well within a huge transaction.
