Castle - Hot-Code Upgrade Support for Elixir

News Announcing
, ,
1

Castle is a package I put together to assist with building Elixir releases capable of being live-upgraded via the ‘official’ release handling mechanisms.

It augments existing Elixir release support, providing:

  • build-time support for appup and relup handling and release generation.
  • runtime support for sys.config generation (incl. support for runtime.exs)

It additionally provides script support for release mgmt, adding commands to unpack, install, commit and remove releases in running systems.

This is an early release, there’s definitely a few rough edges notably:

  • No support for config providers
  • Untested on umbrella apps
  • No Windows support

I will definitely get to the first two in the short term.

Edit - Just to add, it will not generate your appup scripts for you and is unlikely to. If you need hot-code upgrades for your app, it is assumed you understand the implications and will write your own appups.

41 Likes
Library for runtime application configuration—interested?
Distillery: sasl.app does not exist or cannot be loaded
What is the current approach to doing hot code reloading?
Multiple errors when trying to do a basic release with Distillery. Is it OTP 26 compatible?
2

that’s so cool! I am looking for a library for this

2 Likes
3

Great library, I’ve thought about writing something like this after finishing my current projects

1 Like
4

Castle 0.2.2 has been released:

  • Support for config providers is added. Like runtime.exs, Castle takes over the invocation of these providers in a separate instance prior to boot, or in the running instance before hot-code upgrades/downgrades.
  • The generation correctly honours the config_env/0 property during config generation.
7 Likes
5

Very happy to see this effort!

Out of curiosity: would it be possible for libraries to include appup files in their packaging, which can then be used in the generated relups? I’ve seen apps in OTP itself that have them and I was wondering how that worked.

6

I would like to see more maintainers provider appup files with their libraries, but i recognise this is out of scope for many of them. I think the appup generation tooling itself needs to improve as well.

From a technical perspective, it is the responsibility of the build tool to move appup files into the ebin folder of the compiled application. Castle does this currently via its appup compiler.

2 Likes
7

Fully agree with you here. For most libraries like you said it is probably out of scope, but at least having stub appups that just restart the application on upgrade would be a start.

I’ve just tested this, and like you said it is very straightforward to add an appup to libraries: including Castle in the library’s dependencies with the appup compiler and writing the appup is sufficient to get the appup in the ebin of the end application :slight_smile:
Is my understanding correct that in this case, the library can specify runtime: false for Castle, as it only uses it at compile time?

8

Is my understanding correct that in this case, the library can specify runtime: false for Castle, as it only uses it at compile time?

Castle is also used at runtime, specifically, when you upgrade/downgrade a release e.g.

myapp> ./bin/myapp install 1.2.3 # Hot upgrade to release v1.2.3

The shell script sends an RPC into Castle (e.g. Castle.install/1) to prepare the sys.config for that version, prior to asking the release handler to perform the upgrade.

I’m considering whether to split out the build time and runtime parts into separate apps.

1 Like
9

I should have phrased my question a bit better: I meant having runtime: false in the dependency that ships an appup whilst still having it included in the runtime in the final application - in that case my understanding is that the release command will still include Castle into the main release for the app?
In such case if somebody installs the library that doesn’t use hot code upgrades, the app won’t be loaded - so it’s easier to justify adding it as a library dependency.

10

I see you mean. Yes, a library could take Castle only as a build time dependency.

1 Like
11

Castle 0.3.0 has been released:

  • This version separates out all the build-time support (release assembly, appup/relup handling etc) into a separate dependency, Forecastle. Applications should continue to use Castle as their dependency.
    • Mix Projects that don’t define a release, but do wish to generate an appup into their ebin folder, can bring in Castle as runtime: false.
    • Projects that do define a release, should bring in Castle as a runtime dependency. Due to its separation from build-time concerns, the dependency now consists of a single small module.
  • Prior versions of Castle wrapped the existing shell script in the release’s bin folder, which was a kludge. This version replaces the shell script entirely, with the new release handling commands now integrated directly.
4 Likes
Does anyone here deploy using Ansible, Haproxy on baremetal?
12

You you are right .

13

I finally released 1.0.0 of Castle. Some decent improvements:

  • Castle no longer runs via the release script. It now ships a separate castle script in bin that brings the environment in correctly, and through which you can access the unpack/install/commit commands.
  • It does config expansion into sys.config in a separate temporary node
  • It supports upgrades that require emulator restarts.

As before, it still depends on Forecastle for build time support, notably mix tasks for appup/relup generation and checking. It also provides support for testing appups and relups.

4 Likes
14

Very nice. I can’t believe I’ve never heard of this package before! Seems like it could be very useful.

1 Like
15

The documentation states “an external supervisor such as systemd, Docker, Kubernetes or runit for upgrades that restart the emulator” is a requirement. On FreeBSD, we use the simple daemon program for program supervision. Should I do anything specific so I can use castle in this environment where the listed supervisors don’t exist or are not used?

16

I haven’t tested this, but it could work - something like this:

# rc.d
pidfile="/var/run/${name}.pid"
procname="/usr/sbin/daemon"
command="/usr/sbin/daemon"
command_args="-r -R 2 -P ${pidfile} -u my_app -S -T ${name} /usr/local/my_app/bin/my_app start"

-r because if a restart is required, the beam will exit 0.

If that doesn’t work, I’d be happy to take a further look, but I don’t normally run FreeBSD, so might take a day or two to confirm

17

Thank you! I already have supervision working but was wondering if castle does something special, like communicating with the supervisor explicitly. My impression is that it’s not, which is good for me.

1 Like