How do I debug this?

Optimizing ram usage is usually not worth the effort, this is a compromise GC languages have.

There is one thing when ram usage spikes happen and another when there is memory leaking, and judging by your description, you most probably have a spike.

For the record, what are the specs of your machine?