I encountered a similar slowdown while reading a 1.5 GiB log file (which contained enormous NUL byte sequences due to corruption) line by line in Elixir 1.4.2 and Erlang 19.2 under Linux 3.16, which took 8 hours and 7 minutes to complete!
However, I was able to bring the execution time down to 90 seconds (thereby achieving a massive 320x speedup) by reading the file as a raw byte stream (instead of UTF-8) and with generous caching:
file = "path/to/very/large/file"
io = file |> File.open!(read_ahead: 128 * 1024) # 128 KiB cache
lines = io |> IO.binstream(:line)
See my blog post for the details on the major contributing sources of this solution. Cheers!
