Tag Archives: Sidekiq

Avoiding Sidekiq Memory Leaks By Killing It Regularly

Memory leaks are hard to hunt down … but killing things is super easy 🙂
(you have a monitor that restarts them … right !?)

# memory leaks or config changes happen ...
# shutdown -> slowly kill all threads -> restarted by service
# log of all restarts is kept in tmp/shutdown
def trigger_worker_restart_after_interval
  Thread.new do
    interval = 30*60
    time = interval + rand(interval)
    puts "Planning restart in #{time}s -> #{(Time.now + time).to_s(:db)}"

    sleep time
    puts "Planned restart"
    FileUtils.mkdir_p("tmp")
    File.write("tmp/parent_pid", Process.pid)

    # end this thread to ensure a clean exit and kill yourself
    `bundle exec sidekiqctl stop tmp/parent_pid 60 2>&1 >> tmp/shutdown &`
  end
end

Enqueue into Sidekiq via pure Redis (without loading sidekiq)

We want to enqueue jobs, but do not want to blow up the app with sidekiq and it’s dependencies.

Usage

RawSidekiq.enqueue("XyzJob", [1,2,3], :namespace => "custom")

Code

# https://grosser.it/2013/01/17/enqueue-into-sidekiq-via-pure-redis-without-loading-sidekiq
require "json"
require "redis"
require "securerandom"

class RawSidekiq
  def self.enqueue(queue, klass, args, options={})
    payload = {
      'class' => klass,
      'args' => args,
      'jid' => SecureRandom.hex(12),
      #'retry' => true
    }.to_json

    conn = Redis.new
    conn.multi do
      conn.sadd([options[:namespace], "queues"].compact.join(":"), queue)
      conn.lpush([options[:namespace], "queue", queue].compact.join(":"), payload)
    end
  end
end