Category Archives: Server

Tracking Solr Replication Delay via Scout or Sheriff

We now keep track of our solr replication delay, maybe you should too 😉

Works with scout-app or sheriff(free/self-hosted)

class SolrReplication < Scout::Plugin
  needs 'open-uri'

  OPTIONS=<<-EOS
    master:
      default: http://192.168.2.114:8983
    slave:
      default: http://localhost:8765
  EOS

  def build_report
    replication_path = '/solr/admin/replication/index.jsp'
    rex = /Generation: (\d+)/
    master = open(option(:master)+replication_path).read.match(rex)[1]
    slave = open(option(:slave)+replication_path).read.match(rex)[1]
    if master and slave
      report 'delay' => master.to_i - slave.to_i
    else
      error "Incorrect values found master:#{master} slave:#{slave}"
    end
  end
end

Automatic pull, merge and push for git feature branches

We use this script to automatically merge all features into staging and re-merge those features with deploy-ed code. It will stop when there is a merge conflict -> resolve and start it again.

Automatic –>
often -> smaller conflicts.
no thinking/typing -> less mistakes/headaches.

Usage
e.g. store it in scripts folder or turn it into a rake task

scripts/push_and_merge_all
scripts/push_and_merge_all stage # to also deploy everything to staging

Code

#!/usr/bin/env ruby
require 'rubygems'
require 'rake'

sh "git status | grep 'nothing to commit' && echo 'you are clean''" # ensure we are not dirty
current_branch = `git branch | grep '*'`.split.last
sh "git fetch origin" # get up-to-date info about what needs pulling

def merge(branch, options)
  sh "git checkout #{branch}"
  push_and_pull
  options[:with].each do |merged_branch|
    sh "git checkout #{merged_branch}"
    push_and_pull
    sh "git checkout #{branch} && git merge #{merged_branch}"
  end
  push_and_pull
end

def push_and_pull
  status= `git status 2> /dev/null`
  remote_pattern = /# Your branch is (.*?) /
  diverge_pattern = /# Your branch and (.*) have diverged/
  if status =~ remote_pattern
    if $1 == 'ahead'
      sh "git push"
    else
      sh "git pull"
    end
  elsif status =~ diverge_pattern
    sh "git pull && git push"
  end
end

to_stage = ['master', 'feature_a', 'feature_b']
to_stage.each{|branch| merge(branch, :with => ['deploy']) }
merge('staging', :with => to_stage)

sh "cap deploy" if ARGV[0] == 'stage'
sh "git checkout #{current_branch}"

Hash in response + purge.hash for Varnish 2.1

Varnish 2.1 made req.hash unreadable / does not store it. So purge.hash will no longer work.

Getting hash output back

sub vcl_hash{
  set req.http.X-Hash = req.http.host;
  ... more stuff ...
  set req.http.X-Hash = req.http.X-Hash "#"; # add trailing '#' to be compatible to varnish 2.0
  set req.hash += req.http.X-Hash; # store in unreadable req.hash
  return(hash);
}

sub vcl_deliver {
  ... more stuff ...
  # store hash in response for matching / testing that hashing works
  set resp.http.X-Hash = req.http.X-Hash;
  return (deliver);
}

Purging on hash

varnishadm -T 127.0.0.1:6082 'purge req.http.X-Hash ~ something'