Standard Libraries vs Tail-Recursion

Questions & Help Questions
,
8

Thanks all for the answers! I updated the code with some of your advices. It runs so efficiently now!

Here you can see it:

defmodule RepeatedWords do

  def word_count(readFile) do
     readFile
     |> File.stream!
     |> sanitize
     |> count
     |> order
  end

  defp sanitize(words) do
    words
    |> Stream.flat_map(&String.split(&1, ~r/(*UTF)\W/, trim: true))
    |> Stream.map(&String.downcase(&1))
  end

  defp count(words) do
    Enum.frequencies(words)
  end


  defp order(words_count) do
    words_count
    |> Enum.sort_by(&(elem(&1, 1)), :desc)
  end

end

I will still work more updates onto this, but im fascinated for the memory reduction with these changes (went from 3gb memory on large file to not going up of 25mb no matter what).