In the Usage Guidelines section of the GenStage doc (GenStage — gen_stage v1.3.2), the following paragraph got me thinking:
As you get familiar with GenStage, you may want to organize your stages according to your business domain. For example, stage A does step 1 in your company workflow, stage B does step 2 and so forth. That’s an anti- pattern.
The same guideline that applies to processes also applies to GenStage: use processes/stages to model runtime properties, such as concurrency and data-transfer, and not for code organization or domain design purposes. For the latter, you should use modules and functions.
Can someone please tell me if I am thinking about the following properly:
I have a stream of events coming in (the producer of events is an external application). An event is a json and there are multiple types of events. Depending upon the type of event, certain checks need to be performed on the fields within the json, for example ‘quantity’ > 1000. Also, certain checks need to be performed regardless of the event type. Another point is that some of the checks based on a time window, for example, counting the number of orders with ‘quantity’ > 1000 in the last 10 minutes.
After reading about GenStage, it felt like the right approach to use, and I was thinking of the pipeline like this:
A (gets the events from an http stream) → B (splits the streams based on event type) → C1, C2, C3 (these subscribe to the stream based on the event type and process the event).
Does this sound like the right approach? Or, is this the anti-pattern that the GenStage document describes. Please help me see this properly.
