ActiveMemory v0.8 — in-memory tables that speak Ecto (schemas, changesets, Repo-style reads)

News News & Updates
, , , , , ,
1

ActiveMemory 0.8.0 — in-memory tables that speak Ecto

ActiveMemory 0.8.0 is now on Hex.

ActiveMemory is an in-memory store built on ETS and Mnesia: typed records you query by any attribute, not just a key — no cache keys to design, no match specs to hand-write.

The 0.8 release has one goal:

If you know Ecto, you already know ActiveMemory.

Links:

A table can just be an Ecto schema

defmodule MyApp.Planet do
  use ActiveMemory.Table, type: :ets

  use Ecto.Schema
  import Ecto.Changeset

  @primary_key {:uuid, Ecto.UUID, autogenerate: true}

  embedded_schema do
    field :name, :string
    field :gravity, :decimal
    timestamps()
  end

  def changeset(planet, attrs) do
    planet
    |> cast(attrs, [:name, :gravity])
    |> validate_required([:name, :gravity])
    |> validate_number(:gravity, greater_than: 0)
  end
end

defmodule MyApp.Planet.Store do
  use ActiveMemory.Store, table: MyApp.Planet
end

Add the store to your supervision tree and the table exists — no migrations required.

Table metadata is derived from the schema. Autogenerated primary keys and timestamps() are filled on write. Since this is a real Ecto schema, existing changeset functions work as expected.

write/1 takes a changeset directly, similar to Repo.insert/2:

def create_planet(attrs) do
  %MyApp.Planet{}
  |> MyApp.Planet.changeset(attrs)
  |> MyApp.Planet.Store.write()
end

# {:ok, %Planet{}} | {:error, %Ecto.Changeset{}}

And the reads are the ones your fingers already know:

{:ok, planet} = Store.get(uuid)

planet = Store.get!(uuid)
# raises ActiveMemory.NotFoundError

{:ok, planet} = Store.get_by(%{name: "Mars"})

Store.count()
# O(1), asks the table for its size

Store.exists?(%{name: "Mars"})

Store.all(
  order_by: {:desc, :inserted_at},
  limit: 20
)

Store.select(
  match(:gravity > threshold and :name != "Pluto")
)

If you would rather not use Ecto, the original attributes DSL still works and now supports types:

attributes do
  field :name, :string
  field :gravity, :decimal
end

Those types feed casting and validation.

The parts that aren’t Ecto imitation

These are the reasons ActiveMemory exists instead of just being an Ecto-style wrapper:

Atomic withdraw

withdraw/1 finds a record, deletes it, and returns it in one atomic operation.

  • ETS uses :ets.select_delete/2
  • Mnesia uses a transaction

Under concurrent access, exactly one caller gets the record.

Everyone else gets:

{:error, :not_found}

This is useful for:

  • one-time tokens
  • magic links
  • 2FA codes
  • single-use jobs

No Lua scripts. No FOR UPDATE SKIP LOCKED.

TTL support

Give a table a TTL and every record gets a lifetime.

Expired records are never returned from reads, and the store sweeps them to reclaim memory.

defmodule MyApp.MagicLink do
  use ActiveMemory.Table,
    type: :ets,
    ttl: :timer.minutes(15)

  attributes do
    field(:token, :string)
    field(:user_id, :integer)
  end
end

A complete one-time token flow:

case MyApp.MagicLink.Store.withdraw(%{token: submitted}) do
  {:ok, link} ->
    log_in(link.user_id)

  {:error, :not_found} ->
    reject()
end

Crash resilience

ETS tables are owned by a heir process, so a store crash does not destroy the data. The restarted store reclaims the table.

ETS and Mnesia behind one API

Start node-local with ETS. Move a table to replicated Mnesia by changing configuration.

The Mnesia partition trade-offs — including majority: true quorum writes — are documented instead of hidden.

What it is not

Being upfront, because this audience will ask the right questions:

ActiveMemory is not a replacement for your database.

  • Not a system of record.

    ETS data disappears with the node. Mnesia only persists when using disc_copies.

    Keep durable data in your database. ActiveMemory is for things like:

    • sessions
    • tokens
    • feature flags
    • configuration
    • API keys
    • reference data

  • Attribute queries scan.

    This is intended for small-to-medium tables. Secondary indexes that queries actually use are the headline feature planned for 0.9.0.

    limit/offset are convenience pagination, not indexed pagination.

  • write/1 is currently an upsert.

    There is no insert/update distinction yet. update/1 and atomic counters are planned for 0.9.0.

    Also not currently supported:

    • associations
    • cross-table transactions on ETS

  • Upgrading from 0.7

    Two behavior changes:

    1. one/1 and withdraw/1 now raise ActiveMemory.MultipleResultsError when multiple records match, matching Repo.one.
    2. Ecto schemas whose primary key is not the first field now fail during table creation because the first field is the table key.

    Details are available in the changelog.

Roadmap

The 0.9.0 roadmap currently includes:

  • test isolation/sandbox support
  • secondary indexes
  • update/1
  • atomic counters (rate limiting!)
  • telemetry
  • introspection

The full list is available under Planned Enhancements.

Feedback on priorities is genuinely welcome, especially from anyone running Mnesia in production.

Happy to answer questions about the internals — match-spec compilation, the heir mechanism, or the Mnesia partition behavior write-up that went into this release.

3 Likes