CubDB, a pure-Elixir embedded key-value database

Chat & Discussions Discussions
, ,
47

I want to take the time to thank you for your very insightful post!

I am now a couple of days down the road, and I have used CubDB in the ways you suggested: Having one database containing multiple datastructures so they can be used transactionally across different kinds of data, and I use the min_key/max_key tips you suggest :smile:.

Also, kudos for the way you run streams over the selected records :+1:. The code looks very clean, and I am very happy that memory usage while running a select with a reducer is essentially constant (rather than reading in all records before doing something with them).

I now do came across a bit of an issue. I am using CubDB on my Nerves device, but it seems that not all data is properly kept track of, and some data is lost during reboots:

iex> CubDB.select(MyApp.DB, reduce: {0, fn x, acc -> acc + 1 end})
{:ok, 3543}
iex>  Toolshed.Nerves.reboot
# Wait for Nerves to restart and come back up
iex> CubDB.select(MyApp.DB, reduce: {0, fn x, acc -> acc + 1 end})
{:ok, 1871}

It seems like many records are lost. What is going on here?

The database server is started as follows, and is part of my supervision tree:

CubDB.start_link("/root/data/my_db", [auto_file_sync: true, auto_compact: true], [name: MyApp.DB])

(Or to be exact, the app contains a module which has the following child_spec definition to allow it to be added to the supervisor using just MyApp.DB:

defmodule MyApp.DB do
  def child_spec(_) do
    %{id: __MODULE__, start: {CubDB, :start_link, [Application.get_env(:my_app, :my_db_location, "data/my_db"), [auto_file_sync: true, auto_compact: true], [name: __MODULE__]]}}
  end

  # ... some other helper methods that use CubDB.get/fetch/put under the hood.

end

)

EDIT: This particular code snippet is outdated as newer versions of CubDB expect different arguments to start_link. @BrightEyesDavid posted an updated snippet below.

1 Like