I’m having a very frustrating issue debugging something, and I’m now resorting to the forum for help. I’m sure people will recognize this question as I’ve exhausted a lot of options trying to solve it. I have an API that currently gets ~100,000 incoming requests/minute (1,600 RPS). For each incomi…

Based just on reading through this, nothing immediately stands out. My main purpose in replying is just to say that my experience with kubernetes heavily suggests [image] 9mm: (UNLESS each instance of erlang is somehow allocating outside resources based on a limit, and creating new app instance…

As hinted at by the first reply, when scaling Elixir you’ll always want to first increase the number of CPUs available per pod before you scale the number of pods. There’s no point in running 12 1 cpu pods when you could just do 1 pod with 12 cpus. The latter will perform much better. EDIT: When yo…

This is pretty far outside what I know so the following is just a theory… What do you see when you run ss -s in the container to get a summary of the socket states when the problem is happening? My (probably wrong!) theory is that you’re running out of space in a NAT table somewhere. You’re connect…

I don’t believe I have any “limits” imposed. Is that in my YAML file? I will double check if you tell me that’s where they are. Here is my results from inside the docker container: bash-4.4# ulimit -a core file size (blocks, -c) unlimited data seg size (kbytes, -d) unlimited sc…

Interesting. I’m using “Google container-optimized OS” and that command isn’t available. Is that something I can install with apt-get or does it have another name? I don’t have it on my mac either so I can’t read about it

Definitely do a test where you explicitly set 12 CPUs and 24 schedulers in your vm.args. In your pod spec do: limits: cpu: 12000m requests: cpu: 12000m and in your vm.args +S 24. One aside to check here if you’re on K8s is DNS lookups. Each outbound request from within K8s will do li…

Awesome suggestions! Let me check some of this stuff. So you’re saying on the domain thing that the domain I give to the HTTP lib is like this? http://.example.com?

Oddly enough that works on my mac (huge output) but when I run it on the server the command works but with different options: netstat: unrecognized option: s BusyBox v1.28.4 (2018-12-06 15:13:21 UTC) multi-call binary. Usage: netstat [-ral] [-tuwx] [-enWp] Display networking information -r Rout…

Yes that’s correct. It’ll still hit KubeDNS but it lowers the number of different domains that the KubeDNS service will try to resolve. All of my other comments about KubeDNS still apply though, this is just a handy optimization.

Elixir Programming Language Forum

Difficult debugging problem

Questions & Help Questions

chrismcg December 26, 2018, 9:57pm 8

maybe netstat -s is available?