I’ve tried a few things and have better performance. Much smoother/consistent response cycles. The big changes, for my scenario, were:
- Disabling of hackney’s pooling (
[{:hackney, [pool: false]}]), since my request workers are already managed by a pool - Drastically lowered the number of request workers in my pool. At one point I had over 1000 workers that could concurrently send out requests. When I knock that down to even 100 after removing the pooling, I noticed a much more consistent HTTP request cycle.
I also made some SSL-specific changes to match some old code I had in Ruby with Mechanize. I do not know if this had any impact, but it was worth a shot and it hasn’t hurt my performance so far.
- Modified the SSL versions allowed -
opts = [{:ssl, [versions: [:tlsv1, :"tlsv1.2", :"tlsv1.1"]]} | opts] - Allow SSL requests to fail verification -
opts = [{:hackney, [insecure: true]} | opts]
Having said all of that, I’m not suggesting anyone follow what I’ve done. Ignoring SSL verification is probably not the best idea. Also, I’m guessing that hackney has pooling for a good reason. But it does seem to have helped me to just bypass it in my scenario. If I were not managing a request pool myself, I’d say its critical to leave the hackney pooling on or else you’d likely overwhelm the system.
