Yeah, the point was to optimize the code algorithmically. Having established that most of the time is spent in the code which decides whether the line should be filtered or not, I tried to reduce the processing there. We don’t need to process the entire line if we’re deciding on the first column only. This had some savings in your csv example, but could have saved a lot more for larger rows. Your trick with checking the last digit is also a great save, since we don’t have to collect all the digits, nor convert to int.
So now, we’re better than Ruby, although it would be interesting to see how Ruby would perform with similar optimizations.
Moreover, the :delayed_write puts the streaming version in the area of Ruby (which I suspect buffers by default). So my non-conclusive conclusion would be that for this contrived microbenchmark we can expect similar performance in both languages (assuming both implementations are tuned), so I don’t see problems with Elixir I/O compared to Ruby (which your otherwise great post seems to imply).
