Hello, Why is this Elixir code: defmodule Foo do def run(file_name) do File.open! file_name, [:read], fn f -> IO.stream(f, :line) |> Enum.each(&process_line/1) end end defp process_line(line) do String.rstrip(line) |> String.split(",") end end [ file_name | _ ] = Syst…

Friend sent me this: http://blog.alainodea.com/en/article/398/blazing-fast-concurrent-text-i-o-in-erlang

I assume that accounts for milliseconds, not many seconds… :blush:

I’m not sure how fast BEAM boots up, but have you tried profiling using fprof ?

What version of Ruby is that? Not all of them will use UTF-8 by default. You could also load the whole file into memory first, rather than stream. If I’m not mistaken that’s what the Ruby version does.

build executable with escript? [image] Writing a command line application in Elixir Writing a command line application in Elixir

Running an elixirscript means to compile and then call into the compiled module. I do not think, that for that short script the compiletime will make any significant difference.

[image] NobbZ: I do not think, that for that short script the compiletime will make any significant difference. Yep you are right

on 4mb csv file php is about 10x faster

[image] andre1sk: on 4mb csv file php is about 10x faster Oye, so that’s it, huh. I’m writing a distributed computation framework (basically an Apache Storm rip off) in both Elixir and Ruby. I’m at the point now where I can benchmark and analyze post-computation statistics. As expected, El…

Measuring with time is a bit misleading, since you’re also measuring the time to start the VM. Probably won’t make big difference in this case, but to get a more precise number, use :timer.tc . Also, since chomp and split in Ruby are implemented in C, I’d expect some constant factor difference in Ru…

if you use IO.binstream instead of File.stream it speeds things up x2

Help with performance (file io)

Questions & Help Questions

andre1sk June 7, 2016, 2:12pm 12

Yep you are right