I’m having a very frustrating issue debugging something, and I’m now resorting to the forum for help. I’m sure people will recognize this question as I’ve exhausted a lot of options trying to solve it. I have an API that currently gets ~100,000 incoming requests/minute (1,600 RPS). For each incomi…

Definitely do a test where you explicitly set 12 CPUs and 24 schedulers in your vm.args. In your pod spec do: limits: cpu: 12000m requests: cpu: 12000m and in your vm.args +S 24. One aside to check here if you’re on K8s is DNS lookups. Each outbound request from within K8s will do li…

Awesome suggestions! Let me check some of this stuff. So you’re saying on the domain thing that the domain I give to the HTTP lib is like this? http://.example.com?

Oddly enough that works on my mac (huge output) but when I run it on the server the command works but with different options: netstat: unrecognized option: s BusyBox v1.28.4 (2018-12-06 15:13:21 UTC) multi-call binary. Usage: netstat [-ral] [-tuwx] [-enWp] Display networking information -r Rout…

Yes that’s correct. It’ll still hit KubeDNS but it lowers the number of different domains that the KubeDNS service will try to resolve. All of my other comments about KubeDNS still apply though, this is just a handy optimization.

Hey everyone I’m going through these replies slowly, sorry if I don’t respond to you right away. I am 1000% grateful and thankful for all your guys and gals input/feedback. I’m processing all of it currently, and it’s taking me awhile to research each reply In my pod YAML this is what was there: r…

Phoenix has been tested on boxes up to 128 cores and it scaled linearly, so 64 should be fine. I’d consider manually specifying the number of schedulers because I don’t know off the top of my head if erlang can tell what you’ve allocated for the pod or not. Usually you want twice as many schedulers …

Ok I actually tried running it on the instance itself (via SSH) rather than using bash IN the container: Ip: Forwarding: 1 416024035 total packets received 2 with invalid addresses 415262118 forwarded 0 incoming packets discarded 761912 incoming packets delivered 4160034…

Awesome thanks!! And btw I still havent read your other reply I know it was a really good reply (abou tthe schedulers and vm.args) so I’m saving it a bit to do some of these easier replies first. This is such good info, thank you I am eternally grateful :folded_hands:t3:

Awesome! I will read those – but just to be clear would it be OK to put it at 64000m? or even “no limit” (can I delete the entire resource line)? I was testing different instance sizes and I know it wont even schedule a pod into a node w/o enough resources. It would be great to move the limit entir…

Difficult debugging problem

Questions & Help Questions

outlog December 26, 2018, 10:58pm 14

looks like you found the reason! that is 0.1 cpu per pod.. explains your test results.

read

and