Gulping millions of lines into a string and then splitting works, but the point of streaming by line is to avoid unnecessarily building a humongous stream.
This is actually how the ports are implemented (and hence System.cmd). Internally, port just reads everything that is available on the stdout buffer and send it as a message to the calling process, it does not wait for the caller to consume. Also, the config {line, L} how the output is delivered to calling process not how it is read into VM. So if your external program producing a lot of output in short amount of time then it still can get accumulated in your beam vm memory irrespective of the line limit config.
You can test it out in iex with observer.
iex(1)> :observer.start
:ok
iex(2)> Port.open({:spawn_executable, "/bin/cat"}, [{:args, ["/dev/random"]}, {:line, 100}, :binary, :use_stdio])
#Port<0.5>
For most of the use case, it is not a big issue. If it is, then there are third party libraries which solves in different ways.
