Getting fast I/O stream processing in Elixir does require some non-obvious tweaks. Due to the way I/O works on the BEAM (there’s a special process that does I/O and passes results to your process as a message), you want the messages to be as long as possible to avoid overheads.
Doing it line by line is about the slowest way possible.
The tricks I have learned are documented in
https://github.com/bbense/beatwc
But in general the bigger chunks you read and process things in the faster it goes. For a 4 meg csv file I would just slurp the whole thing into memory and operate on the resulting binary.
