Nice guide! Reading through it reminded me of something.
I had been considering for some time whether it would be a good idea to make the tuple encodings “follow” Erlang term order. Not exactly, but closer than the FDB tuples. The first step would be to rearrange the typecodes to match term order, which is easy enough.
But the problem is that Erlang tuples are sorted length-first rather than lexicographically, which is very bad behavior for something like FDB. However, during a conversation with @Asd in the Bedrock thread I realized that if you just avoid tuples and use lists instead you don’t have this problem because lists are ordered correctly (by their elements). Which makes perfect sense, because computing the length of a list would be very expensive. (It remains a mystery why tuples are compared in such an unhelpful manner, though.)
The irony is that I was implementing tuple encodings a couple of weeks ago following the :erlfdb_tuple implementation (out of laziness) and I noticed that the encoder/decoder uses lists internally and converts from/to tuples at the start/end, which makes perfect sense because you want to build up the list incrementally as you parse. And I thought “why bother, the lists will generally be short anyway”, so I just used lists in the API instead of tuples.
So as it turns out, I have actually already done this. By accident!
With erlfdb this is probably not something you want to change at this point (and you wouldn’t want to break compatibility with FDB tuples either), but I’m curious what you think about trying to follow term order with the encodings. It’s not something that actually matters in reality, but I find it oddly satisfying.
The performance of lists vs. tuples is an interesting question. In practice, I assume tuples are slightly faster:
[a, b, [c, d]] = Tuple.unpack(bin)
# vs.
{a, b, {c, d}} = Tuple.unpack(bin)
But for short lists I’m doubtful there is a meaningful difference. And using tuples costs some performance too because there is an extra conversion (:erlfdb_tuple builds up a list first). Is there any record of why erlfdb uses tuples over lists?
The integer/float comparison behavior seems like a bad path to go down, though, so I would still deviate from term order there I think.
