Ah … yeah I had an API product for years that was designed to be async. So people would POST jobs with a callback URL. We’d hit that with results, or with an endpoint to gather results when the job was finished. It worked well and people understood why we had to do that based on the type of work we were doing.
Other than that, I would limit concurrency with a pool. So the whole system can have N number of concurrent jobs. You don’t even need poolboy for this … just a Dynamic Supervisor limiting the number of children.
I personally wouldn’t try and base it off of CPU. I’m sure it can be done I guess but I’d start with concurrency limiting I think to play with that. Maybe then have a failsafe based on CPU if you want to push the boundaries a bit more.
