# Persistent\_term for static assets using Plug?

**URL:** <https://forum.elixirforum.com/t/persistent-term-for-static-assets-using-plug/34295>\
**Category:** Discussions\
**Tags:** plug, phoenix, static-assets\
**Created:** [September 10, 2020, 9:06pm UTC](https://forum.elixirforum.com/t/persistent-term-for-static-assets-using-plug/34295 "2020-09-10T21:06:01Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![akoutmos](https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/akoutmos/32/20230_2.png) [@akoutmos](https://forum.elixirforum.com/u/akoutmos)\
**Post date:** [September 10, 2020, 9:06pm UTC](https://forum.elixirforum.com/t/persistent-term-for-static-assets-using-plug/34295/1 "2020-09-10T21:06:01Z")

</div>

I am curious if anyone has experimented with using persistent\_term ([persistent\_term — OTP 29.0.2 (erts 17.0.2)](https://erlang.org/doc/man/persistent_term.html)) for static assets as opposed to reading from the file system? Obviously you would want to do this in production only so you are not constantly cleansing persistent\_term during development. And you would want something in place to hydrate your persistent\_term prior to starting your endpoint. But mostly curious if the idea is crazy or not.

I forked the Plug.Static plug and modified it to leverage persistent\_term and the initial results looked good.

 ![image](https://forum.elixirforum.com/uploads/default/original/3X/e/c/ecf516a33299d7c8c2f0cb0e1bd1ff6aab9bc82e.jpeg)

---

<div class="post-metadata">

**Author:** ![pera](https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/pera/32/10542_2.png) [@pera](https://forum.elixirforum.com/u/pera)\
**Post date:** [September 10, 2020, 10:01pm UTC](https://forum.elixirforum.com/t/persistent-term-for-static-assets-using-plug/34295/2 "2020-09-10T22:01:56Z")

</div>

Nice results. I don’t serve static assets with Cowboy but I am using persistent\_term to persist some computationally intensive function calls (mostly parsing and graph construction). I wrote the following helper macro for this purpose:

```elixir
  defmacro defpersistent([{name, value}]) do
    atom = quote(do: { __MODULE__ , __ENV__.function})
    function = Macro.var(name, nil)
    quote do
      def unquote(function) do
        case :persistent_term.get(unquote(atom), nil) do
          nil ->
            value = unquote(value)
            :persistent_term.put(unquote(atom), {value})
            value
          {value} ->
            value
        end
      end
    end
  end

```

I then have function declarations that look like this:

```elixir
defpersistent function_name: expensive_computation()
def expensive_computation() do
  ...
end

```

---

<div class="post-metadata">

**Author:** ![voltone](https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/voltone/32/24672_2.png) [@voltone](https://forum.elixirforum.com/u/voltone)\
**Post date:** [September 11, 2020, 4:56am UTC](https://forum.elixirforum.com/t/persistent-term-for-static-assets-using-plug/34295/3 "2020-09-11T04:56:45Z")

</div>

Plug.Static, through Cowboy and Ranch, offloads the actual transmission of the data to the OS, using [:file.sendfile/5](https://erlang.org/doc/man/file.html#sendfile-5). I imagine it has to do more work initially than your alternative, but depending on the file size and the achievable transmission rate it may actually be more efficient when the transfer takes longer.

It might be interesting to run some more tests with different file sizes and transmission rates (preferably on a real network interface). Maybe the optimal solution is a mix, serving small files from persistent\_term while falling back to `sendfile/5` for large files.

---

<div class="post-metadata">

**Author:** ![hauleth](https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/hauleth/32/18942_2.png) [@hauleth](https://forum.elixirforum.com/u/hauleth)\
**Post date:** [September 11, 2020, 8:43am UTC](https://forum.elixirforum.com/t/persistent-term-for-static-assets-using-plug/34295/4 "2020-09-11T08:43:32Z")

</div>

> [@voltone](#):
>
> It might be interesting to run some more tests with different file sizes and transmission rates (preferably on a real network interface). Maybe the optimal solution is a mix, serving small files from persistent\_term while falling back to `sendfile/5` for large files.

IIRC there will be no difference as soon as the files are in memory. The difference would be only if the files are served from the FS, as then `sendfile` can work fully within kernel, without going through user space.

---

<div class="post-metadata">

**Author:** ![voltone](https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/voltone/32/24672_2.png) [@voltone](https://forum.elixirforum.com/u/voltone)\
**Post date:** [September 11, 2020, 8:50am UTC](https://forum.elixirforum.com/t/persistent-term-for-static-assets-using-plug/34295/5 "2020-09-11T08:50:20Z")

</div>

> [@hauleth](#):
>
> IIRC there will be no difference as soon as the files are in memory. The difference would be only if the files are served from the FS

Isn’t that what we’re comparing here: Plug.Static serving up a file, versus an alternative that serves content from memory, in this case `persistent_term`?

---

<div class="post-metadata">

**Author:** ![akoutmos](https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/akoutmos/32/20230_2.png) [@akoutmos](https://forum.elixirforum.com/u/akoutmos)\
**Post date:** [September 11, 2020, 1:47pm UTC](https://forum.elixirforum.com/t/persistent-term-for-static-assets-using-plug/34295/6 "2020-09-11T13:47:36Z")

</div>

I’ll put together some more tests tonight with files larger than 1MB and see how things behave. I’ll also throw this on a separate server so it’s not on local host. Will report back with results!

---

<div class="post-metadata">

**Author:** ![hauleth](https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/hauleth/32/18942_2.png) [@hauleth](https://forum.elixirforum.com/u/hauleth)\
**Post date:** [September 11, 2020, 1:56pm UTC](https://forum.elixirforum.com/t/persistent-term-for-static-assets-using-plug/34295/7 "2020-09-11T13:56:00Z")

</div>

Ahh, I missed that part. In such case `Plug.Static` with `sendfile` should be faster in almost all cases (assuming that the storage is quick).

---

<div class="post-metadata">

**Author:** ![voltone](https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/voltone/32/24672_2.png) [@voltone](https://forum.elixirforum.com/u/voltone)\
**Post date:** [September 11, 2020, 2:12pm UTC](https://forum.elixirforum.com/t/persistent-term-for-static-assets-using-plug/34295/8 "2020-09-11T14:12:39Z")

</div>

Well, if the socket can accept the entire file contents at once, without blocking, then it might be faster to just dumpt the data into the socket from memory, rather than do the system call dance with `sendfile`.

But if the file is larger than what the socket can handle instantaneously, causing it to block on write, then the BEAM as to go off and do other work, and come back later when the socket is ready for more. That is overhead that you don’t get with `sendfile`. This is why I suggested to test with different file sizes, and why testing over the loopback interface is not realistic, since it probably has a humongous TCP window size, so the socket will never block for writing.

---

<div class="post-metadata">

**Author:** ![derek-zhou](https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/derek-zhou/32/19943_2.png) [@derek-zhou](https://forum.elixirforum.com/u/derek-zhou)\
**Post date:** [September 11, 2020, 2:19pm UTC](https://forum.elixirforum.com/t/persistent-term-for-static-assets-using-plug/34295/9 "2020-09-11T14:19:11Z")

</div>

Interesting! For purely static assets, we can use a front-end server like nginx. However there could be some quasi-static pages that change infrequently but are visited frequently, maybe we can pre-render them and stash them here?

---

<div class="post-metadata">

**Author:** ![al2o3cr](https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/al2o3cr/32/3457_2.png) [@al2o3cr](https://forum.elixirforum.com/u/al2o3cr)\
**Post date:** [September 11, 2020, 3:11pm UTC](https://forum.elixirforum.com/t/persistent-term-for-static-assets-using-plug/34295/10 "2020-09-11T15:11:58Z")

</div>

Another potential performance pitfall with large files: getting the data from `persistent_term` to the socket is going to cost at least one copy of the data (from the BEAM to the kernel) where `sendfile` is only transmitting a “get the data from over there” message.