Hi all, big Elixir fan (and newbie!) dropping by to ask something that has been puzzling me for a bit. I’m building an app that relies on Oban for job processing. However, some jobs are getting killed/crashing with obscure errors by the BEAM. These are the strange behaviors: Oban job gets kille…

No, the whole VM is unloading the code that was there before, and then loading the new compiled code. The best thing to do is just let Oban retry the job after the crash.

I’m throwing a dart in the dark here, but I’ve been bitte by HttPoison/hackney pools before, where if the request crashes it doesn’t release the connection, starving the pool in a short time. Try setting pool: false and see if that helps maybe?

Hey, thanks for the idea. I never thought of this – this is much easier to test in isolation. I’ve performed a simple load test, and everything I can throw at it seems to work. The job executes well. I’ll continue testing the worker in isolation and report back – I’m hoping to get a better stack tr…

I figured it could be caused by HTTPoison pools – will absolutely try this, as I have no clue where the process is getting sent a :killed. Just found that pools limit the amount of concurrent connections you can execute, will disable them. This could be the problem: too many Oban jobs are executing…

normally you would wait for a connection to become available, but when you have the issue of having the connections not being released back into the pool you basically loose capacity silently until things blow up. The worst thing is that since it’s very easy to just leave the :default pool, you can…

I’ve disabled the connection pool and wrote a small testing Mix script to load test the behavior. I’m seeing no :killed yet, but will continue testing over the following days under many conditions (including starving the system of memory). Reporting back in a bit. @impl Mix.Task def run(_) do …

I’ll continue testing over the next couple of days. I got some errors, but because the system ran out of file descriptors (I bombed the system with requests). I also saw a Killed error somewhere, so I suspect the problem persists. I will continue testing in different system conditions, such as high…

config :logger, handle_otp_reports: true, handle_sasl_reports: true You’ll probably see the other error with those on.

[image] mattei: Enum.each(1..100, fn _ -> Parcel.Jobs.DeliverWebhook.perform(job) end) This is not concurrent. Your other snippet with Task.Supervisor.async is the way to go. Now, it can be actually pretty fast. What I would do is to simulate load by adding a sleep(1 second) in the server th…

The problem is primarily when using Oban, yes. I will add a Process.sleep at the controller and the Oban job to simulate load :+1: EDIT: Okay, so what ends up happening under the load script is that the server locks up and requests don’t go through. The error persisted. [requests start locking u…

Well this is beyond my knowledge but could it be that your HTTP library maintains long-lived connections that it believes to be still open but are actually closed because the laptop went to sleep, so when it tries to use them again it fails because it could not handle the closed event in a “normal” …

Debugging obscure process crashes?

Questions & Help Questions

mattei May 5, 2022, 5:37pm 17

In that case, is there any way to exclude Oban from hot reloads?