Ruby seems to always grow and then hit the memory limit … which triggers a SIGKILL … which means shutdown without cleanup.
This snippet helps to keep memory low and softly kills the process when memory gets to close to the limit.
# run GC periodically to reduce memory and report to airbrake if we run out (instead of kubernetes silently failing) Thread.new do loop do sleep 60 GC.start used = Integer(File.read('/sys/fs/cgroup/memory/memory.usage_in_bytes')) / 1024 / 1024 max = Integer(`cat /sys/fs/cgroup/memory/memory.stat | grep hierarchical_memory_limit`.split.last) / 1024 / 1024 puts "Ram: #{used}M / #{max}M" raise "Out of memory #{used}/#{max}" if used + 5 >= max end end
A fun little tool that might help too: preoomkiller
One thought on “Ruby on Kubernetes: Memory GC OOMKilled”