StreamData: data generation and property testing for Elixir

Elixir News & Announcements Elixir News
10

Thanks so much for all the links, they were very interesting reads! :slight_smile: The Hypothesis approach is indeed both elegant and powerful but as far as I can tell, it’s not more powerful than the StreamData approach (which as I mention in the README is really Clojure’s test.check approach, credit where credit is due :smiley:).

One thing to note: StreamData is not type-based in any way. It just happens that I defined a bunch of type-ish generators because they are useful :slight_smile:.

I’ll write a blog post about it because it’s very interesting IMO, but the gist is:

  • all StreamData generators take seed as the argument and can use that seed to produce whatever they want. This is analogous to the randomness of the byte stream in Hypothesis.
  • a StreamData generator (which is substantially a function) generates a “lazy tree” when generating a value: a lazy tree is basically a tree where the root is realized (that is, eager, not lazy) and the children are a lazy stream of lazy trees. In StreamData, the root is the generated values and the child subtrees are the shrinks of that value. An easy example is integers: int/0 returns a tree where the root might be 4, and the children might be 0, 2, 3 and 2 might have the child 1 and so on. Shrinking becomes a matter of a somewhat fancy depth-first search in this tree. This also solves exactly the problem that the author of Hypothesis highlights in Hypothesis.
  • The seed that generators accept as argument can be “split”, meaning we can get two seeds from one seed in a deterministic way. So generators just split seeds when they need to pass down the seed. Since everything is deterministic, to “replay” a run you can just pass the same seed (this is already done to integrate with ExUnit where we use ExUnit’s seed).

Last but not least, binary/0 does not have any problem in shrinking both bytes and the length of the binary. There is actually more work done in order to make it shrink like it does today. Whether it’s the right API or not, it’s a different discussion but I’m very much interested in it so please open up an issue in the StreamData repo and we can discuss with everyone else as well :slight_smile:

Hope this made it somehow clearer, as I said I’ll try to put together a blog post because this is interesting and I spent the last months grokking it so I’d be very happy to share the insights!

6 Likes