Setting up proper uniqueness options for Queue in Oban

[sorentwo]

@rindek

However, when running tests, I noticed that if I enqueue the same worker with the same arguments 15 times, it starts executing 10 workers and puts 1 in the available state. I expected it to execute 1 worker and also put 1 in the available state since it’s the same worker and arguments.

In your worker configuration the states list doesn’t include executing, so an executing job is no longer considered unique and you’ll end up with multiples.

@rindek

I’m running multiple different workers for the same queue, so I can’t rely only on args or only on the worker; I need to rely on both.

That’s fine, global partitioning by worker and args is possible. However, I suggest using an explicit list of keys whenever possible for predictability and performance.

[allowed: 1, partition: [:worker, args: :some_key]]

@rindek

Is it possible to configure Oban in this way? If so, what adjustments do I need to make?

Based on your criteria it sounds like you might want to use the Chain worker rather than uniqueness.

Chain workers link jobs together to ensure they run in a strict sequential order. Downstream jobs won’t execute until the upstream job is completed, cancelled, or discarded. Behaviour in the event of cancellation or discards is customizable to allow for uninterrupted processing, holding for outside intervention, or cascading cancellation.

Jobs in a chain only run after the previous job completes successfully, regardless of snoozing or retries.

Chains use the same partitioning format as queues, so optimally you’ll match the options to ensure only one job in a chain runs at once. There’s more in the Optimizing Chains section of the module docs.