Even at 100 messages a second kafka is way overkill. You’re still easily in Postgres territory, perhaps with some offloading of long term timeseries data storage to something like clickhouse. If you want to use some sort of queue / buffer then I’d still be reaching for like SQS or Google Cloud PubSub instead of the headache that can be managing kafka. Just shoving each payload into Oban through completely fine, we run thousands of jobs per second through Oban just fine.
Regarding managing the devices again unless I’m missing something 10k connections is well within the bounds of even a single normal sized phoenix node.
