Difficult debugging problem

I managed to deploy to a plain 64 CPU instance in amazon. I hand installed everything and ran it in foregroud in production mode via ssh.

Still same problem, although it got to about 50K total before dying, that is ~5-10% better but I can see when it starts to fail it looks like this: