Monitor Dalli Connection Changes + Failures

Memcached servers changing leads to a tiny split-brain scenario since some servers might read from different caches then the others … good to keep an eye on it and alert when it happens too often. Here is a tiny snippet to report when it happens.

# config/initializers/dalli.rb
# Whenever the alive-ness of a server changes we read keys from a different server
# which leads to stale keys on the old server and cache-misses on the new servers
# so this should not happen often
# see lib/dalli/server.rb
#
# reproduce: rails c + Rails.cache.get + zdi memcached stop & start
Dalli::Server.prepend(Module.new do
  def down!
    $statsd.increment "dalli.connection_changed", tags: ["state:down"] unless @down_at
    super
  end

  def up!
    $statsd.increment "dalli.connection_changed", tags: ["state:up"] if @down_at
    super
  end

  def failure!(*)
    $statsd.increment "dalli.failed"
    super
  end
end)

Bundler + Rails: realpath expand_path and symlinked vendor/bundle cause ‘already loaded’ errors

Symlinked vendor/bundle results in double load errors since:

  • rails adds the realpath of each engines lib (and various other folders) to the $LOAD_PATH
  • bundler adds the symlinked version to the $LOAD_PATH
  • require_relative uses the realpath

which looks like: `already initialized constant Arturo::Middleware::MISSING_FEATURE_ERROR`

Reproduce:

  • bundle install –path vendor/bundle
  • mv vendor/bundle tmp
  • cd vendor && ln -s ../tmp/bundle bundle
  • enable eager_load + preload_frameworks in config/environment/development.rb
  • rails runner 1

Ruby issue

Patch:

# Gemfile
require_relative 'lib/bundler_realpath'
 
# lib/bundler_realpath.rb
# https://grosser.it/2017/08/19/bundler-rails-realpath-expand_path-and-symlinked-vendorbundle-cause-already-loaded-errors
Bundler::Runtime.prepend(Module.new do
  def setup(*)
    super
  ensure
    linked_bundle = File.expand_path(Bundler.bundle_path)
    real_bundle = File.realpath(Bundler.bundle_path)
    if linked_bundle != real_bundle
      $LOAD_PATH.each_with_index do |path, i|
        $LOAD_PATH[i] = path.sub(linked_bundle, real_bundle)
      end
    end
  end
end)

Rails 5.1 do not compile asset in test vs asset is not present in the asset pipeline

We don’t want to compile assets during test runs, since that is slow, but we also don’t want the asset pipeline to fail because assets are missing.

This will not work if you plan on doing javascript integration tests, but everything else should work fine.

Rails 5.1 added a flag for this which prints deprecations and will be removed in rails 5.2 so that is not a elegant solution either.

config.assets.unknown_asset_fallback = true

So we are now using this fix to fake assets being available!:

# config/environments/test.rb
# make our tests fast by avoiding asset compilation
# but do not raise when assets are not compiled either
Rails.application.config.assets.compile = false
Sprockets::Rails::Helper.prepend(Module.new do
  def resolve_asset_path(path, *)
    super || path
  end
end)