In your solution, you wait for 2 seconds or 400 events, whichever comes first. That means that there might be some fixed latency penalty if messages don’t arrive frequently. There is also a question of these numbers: how did you ensure they are neither too big nor too small?
In my proposal, the difference is that you write as soon as the writer is available. Therefore, if the events arrive less frequently, there will be no penalty. If the events arrive too frequently, the buffer will expand to compensate for that.
Let’s see an example. Consider a write which takes 100ms regardless of the data size. Let’s say that events arrive every 200ms. The writer is always available when the event arrives, so you write immediately (no latency penalty).
Now let’s say that events arrive every 10ms. The first event arrives, you write it immediately. Now, for the next 100ms, the writer is busy, so you’ll buffer next 10 events. Then, when the writer reports back that it’s available, so we’ll write the buffered 10 events at once. The buffer size adapted to the incoming load.
