IO Bottlenecked Processing

I’m confused by all the G/gb/GB mb/MB etc being thrown around.. maybe just stick to one unit, or make sure they are used correctly..

Just wanted to point out that storage speed might be a/the bottleneck as well.. the m5.24xlarge have EBS instance storage, so since this instance has plenty of memory, maybe go straight to memory and not to disk/EBS..

(even if you use an instance with dedicated ssd, I doubt it can hold up to ingressing 25 Gbit straight to disk..)