Well, that’s kind of my point with this project – different usecases will have different requirements. So a library that tries to cover all the usecases is not necessarily a good idea. But some reference code can be really handy, so you can copy the bits you need and adapt it as needed.
Well no, it’s a data stream processing framework. So for processing a message queue, it’s pretty useful. It’s certainly not the only way to ingest a queue, though. A simpler GenServer for queue processing can be found here in my CQRS patterns repo, and it’s pretty much a drop-in replacement for the Broadway-based processor.
Ultimately, anything could call Messaging.process_message_queue_batch/2, so the world’s your oyster.
The main advantage of Broadway in this message queue example is concurrency – it decouples the ingestion from the processing, so one process can pull a batch of 100 messages, and distribute them across a pool of 5 workers. And of course if the work calls for it, Broadway has features such as batching and pre-fetching that could be applied very easily on top of this, since the hard work of building the Broadway Producer is already done,
