Hobbes - a low-level distributed database for the Elixir programming language

Hobbes is a database, and is definitely not a library for stateful actors. When I say “OTP primitive”, what I mean is that Hobbes resembles a very sophisticated persistent ETS table. Where you might use :ets to back an in-memory data structure, you would use Hobbes to back an on-disk data structure.

There is a bit more, though, because Hobbes is not simply “on-disk ETS”. (Actually we have DETS for that.) Hobbes is fault-tolerant and will scale to very large datasets across nodes. An ETS table is not fault tolerant (tied to one node) and cannot be (natively) sharded. Also, ETS tables do not have transactions at all.

And so this is why I say it’s like a new OTP primitive, because there has never been an OTP primitive which can do these things. The closest would be Mnesia, but it suffers from small shard sizes and poor consistency guarantees.

Maybe Hobbes could be used to build a library for stateful actors. But that’s just because it is a tool for persisting state in general. For example:

  • A job library like Oban could use Hobbes to persist jobs (they currently use Postgres)
  • An event-sourcing library like Commanded could use Hobbes to persist events (they also seem to use Postgres)
  • A distributed process registry could use Hobbes, taking advantage of its fault-tolerance and strong consistency

But the main reason I wrote Hobbes is to serve my own needs. As you know, I want something that can replace Postgres itself (and S3 and similar tools). Hobbes is essentially an abstraction layer which contains all of the hard problems associated with building a distributed database, so that I can reuse it to solve the “easy” problems each time. Much like Erlang solves the hard problems of distributed scheduling and message passing and so on.

I can then use Hobbes to build, say, a distributed filesystem, or a relational database.

Actually, the relational DB already has a name: it will be called Memex. And Memex will probably be the thing most will take interest in, but it does not yet exist.

Hobbes will always be the “pro tool” for those who want to get their hands dirty, but just not “rolling a database from scratch” dirty :slight_smile:

6 Likes