Help with Durable Server counter demo as a first step

Questions & Help Questions
,
1

I have an idea to fully use DurableServer.

For a long time I had the idea of creating a management system using Phoenix, but there was some piece missing. I didn’t want to use a DB to maintain the state so I could get by with very low resource utilization and a simple infrastructure, but I also wanted to have HA and not lose the data when something breaks.

The idea of using Durable Server with EKV fits perfectly. I can have a single node management system that stores the data in EKV, and I can the expand to a dual active/passive node (using EKV observer mode to duplicate the data but manually changing the mode if I need to start a new server), or even 3-node cluster with HA.

Now I’ve started to explore if what I want is possible. I have created a DurableCounter application that will expand to cover what i need, and I would love some feedback on it:

There are some things that I would love to check that they are true:

  1. I am calling DurableServer.Superviror lookup before I call GenServer.
  2. I have two layers. The one to distribute the information, so you can have more than one browser and they will see the same counter, and another one that is the DurableCounterState. It feels right, but I am duplicating a lot of logic in both layers.
  3. I have configured the durable server with auto_sync: true. I tried without it but you could miss changes if you killed the server after a quick update.
  4. I have added a session_counter, that is a proxy for the transient metadata (i.e. the state of the managed node), but there a need for duplicated code when I convert the state to something digestible in the LiveView

I am happy with the result so far, but I would be very happy if you help me to get a cleaner solution, and make me feel confident that I am using it right.

My code is here:

Note: I will continue with the active/passive version next.

2

I’ve used Claude to suggest some updates, and the code reflects that.

  • Now I share all the context for a key. This will allow me to have the metadata with different keys when I need to do so.
  • Added tests, including killing the server and restarting it so the session_counter is reset.

Now working on the HA for the server.

3

Finding some limitations on Durable Server now that I am trying to work on the HA side of things.
My idea is to have a principal server that is running live, and a backup server that is just watching the original and having a copy of the database (using EKV), so it can become the principal in case of need.
What I am finding:

  • DurableServer does not replicate the same member / observer / client that EKV does. I need an additional abstraction to make this work, so my DurableServers passive servers can become active.
  • I wanted to be able to show observers, so connecting to the principal could show all the observers. So I can show in the interface what other copies of the database are available so I am ready to move if I need to do maintenance or I want to check the configuration.

I wonder if that makes sense for DurableServer and Group. Happy to make some pull requests if it makes sense for Durable Server and Group.

4

What you are building is way more complex than just using a database.

Use the right tools for the job.

5

Can you ellaborate on what you mean with that? I could use a database but I don’t want to use a separate server to run it (I have a version already using the database).

6

DB can be a separate server but it doesn’t have to be.

You can also get a DBaaS which will most likely be the easiest way to achieve your goal. They will also take care of things like backups and (depending on your service) HA is just a click.

If this is just a learning exercise go for it. If this is production data that can’t be lost I wouldn’t mess around and leave it to the pros.

7

I’ve added support for active / passive HA and scripts to easily test it.