Yeah, it’s a 2013 MacBook pro, quad core 2.6GHz, 16MB RAM.
I expect scala will usually perform better on such synthetic small benchmarks. I wonder though how would it look when it’s a more real-life-like scenario, where these plugs do some real work both CPU and I/O bound, and they allocate some memory which needs to be GC-ed at some point.
Also a question: in your Elixir Plug bench, was CPU fully utilised under load?
If you start with a generated Phoenix app, there might be some issues. First, make sure to run the system as an OTP release (using exrm is best). Another important tip is to either increase log level to :warn or decrease Plug.Logger level to :debug. Otherwise, all requests are logged, and there’s a huge amount of requests under load-test. IIRC, Logger will start applying back-pressure when the queue gets bigger, and that might cause the bottleneck, and CPU under-utilisation during increased load.
Other common mistake is running JSON requests through the browser pipeline (use the API pipeline). Finally, if you’re testing locally, don’t use too many threads, because wrk might interfere with the VM.
Just to be clear, my test was very shallow. I had a simple endpoint with one browser req, and one API req, and issued them with 1:1 ratio with wrk for 30 seconds. That got me about 30k reqs/sec and sub-ms times in 99% percentile.
