# Profiling memory usage in given process **URL:** \ **Category:** Questions\ **Tags:** erlang, beam, profiling\ **Created:** [March 14, 2019, 2:23pm UTC](https://forum.elixirforum.com/t/profiling-memory-usage-in-given-process/20878 "2019-03-14T14:23:12Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![hubertlepicki](https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/hubertlepicki/32/6822_2.png) [@hubertlepicki](https://forum.elixirforum.com/u/hubertlepicki)\ **Post date:** [March 14, 2019, 2:23pm UTC](https://forum.elixirforum.com/t/profiling-memory-usage-in-given-process/20878/1 "2019-03-14T14:23:12Z")
I’ve got some requests misbehaving, and processes handling them suddenly allocate a lot of memory, which can crash my Beam VM. I can monitor and detect that, I can also enforce max memory usage by process, that’s not an issue. I would like to, however, know _what_ is eating up the memory. Ideally I’d be able to take a snapshot of memory used by current process, have it dumped to a file that contains Erlang terms, which later I could analyse and find out what’s on stack, what’s on heap etc. Is there something out there I can use to achieve that? ---
**Author:** ![Fl4m3Ph03n1x](https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/fl4m3ph03n1x/32/11709_2.png) [@Fl4m3Ph03n1x](https://forum.elixirforum.com/u/Fl4m3Ph03n1x)\ **Post date:** [March 14, 2019, 2:30pm UTC](https://forum.elixirforum.com/t/profiling-memory-usage-in-given-process/20878/2 "2019-03-14T14:30:51Z")
A few things occur to me as to _why_ that may be happening: ## `cast` them out! Are you using `cast` ? We had an issue sometime ago where we were flooding a process with messages via cast, which caused its mailbox to grow indefinitely causing it to crash the BEAM VM. ## queues Using queues? Queues that grow without boundary (looking at you hackney) misbehave spectacularly, consuming all the system’s memory until a crash is inevitable. > **[Handling Overload](https://ferd.ca/handling-overload.html)** ## Tools you can use If you want a specific tool, then I know of [erlyberly](https://github.com/andytill/erlyberly), which shows you memory and messages passed between processes. You can also find other tools in this discussion: > [@Is there an Elixir editor with Debugger?](https://forum.elixirforum.com/t/is-there-an-elixir-editor-with-debugger/20683/5): > > I mostly use IEx.pry/0 [https://elixir-lang.org/getting-started/debugging.html#iexpry0-and-iexbreak2](https://elixir-lang.org/getting-started/debugging.html#iexpry0-and-iexbreak2) ---
**Author:** ![hubertlepicki](https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/hubertlepicki/32/6822_2.png) [@hubertlepicki](https://forum.elixirforum.com/u/hubertlepicki)\ **Post date:** [March 14, 2019, 2:39pm UTC](https://forum.elixirforum.com/t/profiling-memory-usage-in-given-process/20878/3 "2019-03-14T14:39:19Z")
I already solved the issue as in why and what is happening. It was this bug in absinthe we stumbled upon: [Obscenely high memory usage with deeply nested inputs during Absinthe.Blueprint.Transform.walk · Issue #569 · absinthe-graphql/absinthe · GitHub](https://github.com/absinthe-graphql/absinthe/issues/569) I’d like to know if there’s something that’ll help me out next time tracking down such and similar issues 🙂 ---
**Author:** ![hubertlepicki](https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/hubertlepicki/32/6822_2.png) [@hubertlepicki](https://forum.elixirforum.com/u/hubertlepicki)\ **Post date:** [March 14, 2019, 2:41pm UTC](https://forum.elixirforum.com/t/profiling-memory-usage-in-given-process/20878/4 "2019-03-14T14:41:18Z")
I had a look at [recon v2.5.3 — Documentation](http://ferd.github.io/recon/) and Erlang’s internal tools but I don’t think there’s anything that will allow me to just have a glance at the memory and figure out what exactly eats it up… ---
**Author:** ![OvermindDL1](https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/overminddl1/32/2677_2.png) [@OvermindDL1](https://forum.elixirforum.com/u/OvermindDL1)\ **Post date:** [March 14, 2019, 2:43pm UTC](https://forum.elixirforum.com/t/profiling-memory-usage-in-given-process/20878/5 "2019-03-14T14:43:18Z")
> [@hubertlepicki](#): > > and figure out what exactly eats it up… How ‘exactly’ do you want it? `:etop.start()` is pretty useful (make sure to use a new console) or observer. 🙂 As for what is on the stack… unsure… ---
**Author:** ![hubertlepicki](https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/hubertlepicki/32/6822_2.png) [@hubertlepicki](https://forum.elixirforum.com/u/hubertlepicki)\ **Post date:** [March 14, 2019, 4:23pm UTC](https://forum.elixirforum.com/t/profiling-memory-usage-in-given-process/20878/6 "2019-03-14T16:23:07Z")
That can come in handy although I don’t think it actually does what I want. I think you can do similar thing from observer if you sort processes by memory, but this is cool to have this in case Erlang on the server has no X support. ---
**Author:** ![axelson](https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/axelson/32/26351_2.png) [@axelson](https://forum.elixirforum.com/u/axelson)\ **Post date:** [March 15, 2019, 9:04pm UTC](https://forum.elixirforum.com/t/profiling-memory-usage-in-given-process/20878/7 "2019-03-15T21:04:01Z")
You can run observer locally and connect to a remote node to view its information. Here’s a blog post about it: [Using Erlang Observer on a Remote Elixir Server - Josh Bavari's Thoughts](http://jbavari.github.io/blog/2016/03/11/using-erlang-observer-on-a-remote-elixir-server/) ---
**Author:** ![arnomi](https://forum.elixirforum.com/letter_avatar_proxy/v4/letter/a/e95f7d/32.png) [@arnomi](https://forum.elixirforum.com/u/arnomi)\ **Post date:** [March 17, 2019, 4:12pm UTC](https://forum.elixirforum.com/t/profiling-memory-usage-in-given-process/20878/8 "2019-03-17T16:12:07Z")
Alternatively, have a look at [observer cli](https://hex.pm/packages/observer_cli). Its a terminal version of observer which is fantastic when no X is available (and actually for many cases I prefer its interface over the graphical observer anyways). ---
**Author:** ![hubertlepicki](https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/hubertlepicki/32/6822_2.png) [@hubertlepicki](https://forum.elixirforum.com/u/hubertlepicki)\ **Post date:** [March 18, 2019, 9:01am UTC](https://forum.elixirforum.com/t/profiling-memory-usage-in-given-process/20878/9 "2019-03-18T09:01:22Z")
But to respond to both of you and @axelson, this won’t help me when I precisely know which process is consuming the memory? I would like to know what is consuming the memory, and unless I am missing some functionality, observer does not allow me to have insignt into what’s in the memory itself ---
**Author:** ![NobbZ](https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/nobbz/32/27235_2.png) [@NobbZ](https://forum.elixirforum.com/u/NobbZ)\ **Post date:** [March 18, 2019, 9:16am UTC](https://forum.elixirforum.com/t/profiling-memory-usage-in-given-process/20878/10 "2019-03-18T09:16:17Z")
`:etop` does tell you about the memory consumption of a process. Problem though: If it is not the processes heap + stack but some references to the bin-heap, then you are lost. Space-Leaks on the bin heap are hard to debug due to its ref-counting nature. And even a single `"a"` which was produced by slicing into the gigabyte of JSON file can cause keeping the **full** JSON forever! ---
**Author:** ![hubertlepicki](https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/hubertlepicki/32/6822_2.png) [@hubertlepicki](https://forum.elixirforum.com/u/hubertlepicki)\ **Post date:** [March 18, 2019, 9:33am UTC](https://forum.elixirforum.com/t/profiling-memory-usage-in-given-process/20878/11 "2019-03-18T09:33:56Z")
Do you mean that etop gives me insight into memory consumption of the process (which it does) or can I also get insight into _what_ consumes the memory? I can’t find the later ---
**Author:** ![NobbZ](https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/nobbz/32/27235_2.png) [@NobbZ](https://forum.elixirforum.com/u/NobbZ)\ **Post date:** [March 18, 2019, 10:26am UTC](https://forum.elixirforum.com/t/profiling-memory-usage-in-given-process/20878/12 "2019-03-18T10:26:39Z")
It tells you only how much memory is consumed by a process. I doubt there will be anything that tells you the actual **what**. ---
**Author:** ![hubertlepicki](https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/hubertlepicki/32/6822_2.png) [@hubertlepicki](https://forum.elixirforum.com/u/hubertlepicki)\ **Post date:** [March 18, 2019, 10:35am UTC](https://forum.elixirforum.com/t/profiling-memory-usage-in-given-process/20878/13 "2019-03-18T10:35:30Z")
Alright. The closest thing to what I want to do would be crashing the Erlang VM and producing a dump file that I could analyse but that’s pretty scary thing to do on a production environment, also the resulting dump will not be scoped to the process I identified but to the whole VM (which can be a good or bad thing depening on how you look at it). ---
**Author:** ![arnomi](https://forum.elixirforum.com/letter_avatar_proxy/v4/letter/a/e95f7d/32.png) [@arnomi](https://forum.elixirforum.com/u/arnomi)\ **Post date:** [March 18, 2019, 11:50am UTC](https://forum.elixirforum.com/t/profiling-memory-usage-in-given-process/20878/14 "2019-03-18T11:50:56Z")
There are a few things you can do. Observer (and observer\_cli) allow you to inspect what processes are currently doing. In observer\_cli, if you ordered the processes by memory usage and you want to inspect the first one, simply press 1. You can then, for example, inspect the messages of the process or its state. The same you can do also from iex. To get the state of a process use `sys.get_state(pid)`. And via `Process.info(pid)` (or better `Recon.info(pid)` which disables some of the unsafe inspections) you can look into, for example, current stacktrace, process dictionary, messages etc. ---
**Author:** ![arnomi](https://forum.elixirforum.com/letter_avatar_proxy/v4/letter/a/e95f7d/32.png) [@arnomi](https://forum.elixirforum.com/u/arnomi)\ **Post date:** [March 18, 2019, 11:53am UTC](https://forum.elixirforum.com/t/profiling-memory-usage-in-given-process/20878/15 "2019-03-18T11:53:44Z")
> [@hubertlepicki](#): > > The closest thing to what I want to do would be crashing the Erlang VM I would say, this is never necessary if you already know which process is causing the issue. As mentioned above, you can first start digging into the exact process. One thing to also look into, is garbage collection. You can use `:erlang.garbage_collect` to force garbage collection for processes. Especially if the process is long running, tuning garbage collection may be necessary. ---
**Author:** ![sasajuric](https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/sasajuric/32/991_2.png) [@sasajuric](https://forum.elixirforum.com/u/sasajuric)\ **Post date:** [March 18, 2019, 1:16pm UTC](https://forum.elixirforum.com/t/profiling-memory-usage-in-given-process/20878/16 "2019-03-18T13:16:30Z")
AFAIK, it’s not possible to get what you’re looking for out of the box. To get some clues, you could use a combination of the following: - memory usage of the process - process stack trace - message queue length - total binary memory usage - total ETS memory usage In case you don’t know which process is the offender, you might also need to include `:initial_call` and `:registered_name` (both available via `Process.info`). Coupled with the knowledge of the code, this should help you narrow down the problem. ---
**Author:** ![hubertlepicki](https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/hubertlepicki/32/6822_2.png) [@hubertlepicki](https://forum.elixirforum.com/u/hubertlepicki)\ **Post date:** [March 18, 2019, 2:36pm UTC](https://forum.elixirforum.com/t/profiling-memory-usage-in-given-process/20878/17 "2019-03-18T14:36:28Z")
Right, but if a process is not a nice member of OTP family, as in it’s a Cowboy protocol handler (i.e. process in which Phoenix’s web requests are being handled), and memory is being allocated within the same process there is nothing that will give me insight to the allocated memory. That’s sad, I know Java has tools that can do precisely that, it’s a shame we don’t have the same 🙂 ---
**Author:** ![garazdawi](https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/garazdawi/32/5853_2.png) [@garazdawi](https://forum.elixirforum.com/u/garazdawi)\ **Post date:** [March 18, 2019, 3:02pm UTC](https://forum.elixirforum.com/t/profiling-memory-usage-in-given-process/20878/18 "2019-03-18T15:02:51Z")
Maybe :erlang.system\_info(:procs) is closer to what you are looking for? From it you can get the values that are currently live on the stack. ---
**Author:** ![sasajuric](https://forum.elixirforum.com/user_avatar/forum.elixirforum.com/sasajuric/32/991_2.png) [@sasajuric](https://forum.elixirforum.com/u/sasajuric)\ **Post date:** [March 18, 2019, 4:26pm UTC](https://forum.elixirforum.com/t/profiling-memory-usage-in-given-process/20878/19 "2019-03-18T16:26:33Z")
> [@hubertlepicki](#): > > there is nothing that will give me insight to the allocated memory The stack trace should help you narrow down the problematic code. We actually had a similar situation recently, and we ended up periodically logging the processes with unusually high mem usage, together with their stack traces. In this particular situation that info was enough to completely understand the root cause. I agree that it’s not quite what you’re looking for, but it can help you in understanding the problem. It’s also worth keeping in mind that in a general case excessive mem usage can be caused by other things, such as messages accumulated in the process queue, or refc binaries, or ets tables, so finding the cause is not necessarily as straightforward as logging the current heap of the process, which is why I mentioned a couple of different things in my previous answer. ---
**Author:** ![arnomi](https://forum.elixirforum.com/letter_avatar_proxy/v4/letter/a/e95f7d/32.png) [@arnomi](https://forum.elixirforum.com/u/arnomi)\ **Post date:** [March 18, 2019, 9:08pm UTC](https://forum.elixirforum.com/t/profiling-memory-usage-in-given-process/20878/20 "2019-03-18T21:08:21Z")
> [@garazdawi](#): > > Maybe :erlang.system\_info(:procs) is closer to what you are looking for? From it you can get the values that are currently live on the stack. Is there a way to get the information of `:erlang.system_info(:procs)` for only a single process/port? [Next page](https://forum.elixirforum.com/t/profiling-memory-usage-in-given-process/20878.md?page=2)