Hello there! I have a mix task which grabs some data from a remote API, collects it into a file (a biggish xml), then sucks it in as a stream and processes doing a bunch of text transformations. Now I’m trying to sketch an integration test for the processing part (skipping the getting-the-data par…

You’re in luck, you can simply use StringIO.open/1 and IO.binstream in order to stream any String: {:ok, stream} = "abc\ndef\nghi\n" |> StringIO.open() stream |> IO.binstream(:line) |> #your own stream processing

Thank you. I’ve rewritten my spec to use the suggested approach, however, now I cannot get my tests to finish running. When I try to running it in IEX (create a stream, start splitting it as a binstream and then just print every line using Enum.each), my system just goes down to its knees. Output …

str = """ <page> <title>Acura CSX</title> <ns>0</ns> <id>3161370</id> <revision> <id>772698868</id> <parentid>770005822</parentid> <timestamp>2017-03-28T20:19:55Z</timestamp> <contributor> <username>GreenC bot</username> <id>27823944</id> …

Why not use one of the string functions? I understand you want to get an enumerable of lines from the string. This can be achieved eagerly with String.split(str, "\n") or lazily with String.splitter(str, "\n").

Because String.split/2 and String.splitter/2 will remove the split-points, but the OP said he needs them intact.

This is really odd though, running the code @vivus-ignis posted in IEx completely froze my Ubuntu 16.10 with an i7 after just a couple of lines and I had to do a hard reset. Even more strange is the fact, that this even happens if the last line was the following: stream |> IO.binstream(:line) |> E…

Should I report that to elixir-lang github issues?

I did the experiment as well in an VBox, 6th gen i5 here. The Box had 4 GiB of RAM available, another 4 GiB of swap. I did not reset the Box, but let it run, The Box took all available CPU I assigned (4 cores at 75%) and I was able to see how memory consumption in the box rose, I had htop open in …

I can also reproduce this with Elixir 1.4.2 / OTP 19. For a string with a few lines is works fine, but with more lines it takes more and more wall-clock time per line. Using a progressively longer string, this behavior becomes quite obvious, and it looks very much like O(n^2) type behavior with n be…

Ok .. I think I have found the issue … In every call to get a line, the process returned by StringIO.open does this: defp io_request({:get_line, encoding, prompt}, s) do get_line(encoding, prompt, s) end get_line, in turn call Erlang’s :unicode.characters_to_list. This converts the whole bits…

Elixir Programming Language Forum

Emulate File.stream! for a string variable

Questions & Help Questions

wmnnd May 9, 2017, 6:40am 4

Could you post the code with which you’re trying this?