Setting up proper uniqueness options for Queue in Oban

Questions & Help Questions
,
1

Hi everyone,

I’m currently working on a project using Oban Pro 1.4.0 with Smart Engine, and I need some guidance on configuring a queue with multiple workers. Specifically, I have the following requirements:

  1. The queue should execute a maximum of 10 jobs per node.
  2. It should not execute more than one unique worker with the same arguments globally (i.e., no more than one instance of the same worker/arguments combination should run simultaneously across the entire cluster).
  3. If a worker with the same arguments is already executing, any new job with the same worker/arguments combination should be placed in the available or scheduled state, so it can be processed immediately after the current one finishes.

Here’s what I’ve tried so far:

Oban Queue Configuration:

my_queue: [
  local_limit: 10,
  global_limit: [allowed: 1, partition: [fields: [:args, :worker]]]
]

Worker Configuration:

use Oban.Worker,
  queue: :my_queue,
  unique: [
    fields: [:args, :worker],
    states: [:available, :scheduled, :retryable],
    period: :infinity
  ]

However, when running tests, I noticed that if I enqueue the same worker with the same arguments 15 times, it starts executing 10 workers and puts 1 in the available state. I expected it to execute 1 worker and also put 1 in the available state since it’s the same worker and arguments.

In contrast, if I enqueue 15 different unique worker/arguments combinations, I would expect it to start executing 10 jobs and puts the remaining 5 in the available state.

I’m running multiple different workers for the same queue, so I can’t rely only on args or only on the worker; I need to rely on both.

Is it possible to configure Oban in this way? If so, what adjustments do I need to make?

Thanks for your help!