Testing multiple Gemfiles and Rubies with WWTD (no Appraisal)

Problem
Testing against many gemfiles/rubies is painful, Appraisal offers a somewhat working solution but adds new steps/problems.

Solution
Use WWTD by using the gemspec for dependencies (optional for git: or a gemfiles/common.rb)
(Also no longer runs gemfiles that are excluded in travis.yml)

Example PR

Usage

  • Run tests on all gemfiles: rake
  • Run tests on all gemfiles and all rubies: rake wwtd
  • Run tests on Gemfile: rake test

Code

# Gemfile
source "https://rubygems.org"
gemspec
gem "rails" # newest

# Rakefile
require 'wwtd/tasks'
task default: "wwtd:local"

# gemfiles/rails32.gemfile
source "https://rubygems.org"
gemspec path: "../"
gem "rails", "~> 3.2.18"

# gemfiles/rails40.gemfile
source "https://rubygems.org"
gemspec path: "../"
gem "rails", "~> 4.0.6"

# foo.gemspec
...
s.add_runtime_dependency "rails", ">= 3.2.18"

# .travis.yml
rvm:
  - 1.9.3
  - 2.0.0
  - 2.1.2
gemfile:
  - gemfiles/rails32.gemfile
  - gemfiles/rails40.gemfile
script: "rake test"

Git dependencies
Cannot be specified in gemspec, so we put them in a loadable file.

# all gemfiles
eval(File.read('gemfiles/common.rb'))

# gemfiles/common.rb
gem "foo", git: "https://github.com/bar/foo"

Locking insights: an alternative to redis set nx ex / memcache add

A lock that does not timeout can lead to a standstill and manual cleanup. Simple solution: redis ‘set ex nx’ and memcached ‘add’.

When indefinite locks happen, getting information on why they happen helps to debug the locking mechanism and see if the processes always fail to unlock.

A softer locking approach to receive feedback when locks expire:

Code

def lock
  timeout = 30
  key = 'lock'
  now = Time.now.to_i
  if redis.setnx(key, "#{now}-#{Process.pid}-#{Socket.gethostname}")
    yield
  elsif (old = redis.get(key)) && now > old.to_i + timeout
    logger.error("Releasing expired lock #{old}")
    redis.delete(key) # next process can get the lock 
  end
end

Not 100% safe since the delete could cause multiple processes to get a lock, but depending on your usecase this might be an ok tradeoff.

Allowing travis to git clone by adding a public ssh key

My new on demand gem server wants to test on travis with ssh cloning, so let’s add a ssh key (from user ‘some-public-token’ that I created and does not have any projects)

#.travis.yml
before_script:
 - echo -e "Host *\n\tStrictHostKeyChecking no\n" >> ~/.ssh/config
 - echo -e "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDkTcgXnHuqR0gbwegnr9Zxz4hTkjjV/SpgJNPJz7mo/HKNbx0rqjj1P0yGR053R9GSFFim2ut4NK9DPPUkQdyucw+DoLkYRHJmlJ4BNa9NTCD0sl+eSXO2969kZojCYSOgbmkCJx8mdgTwhzdgE/jhBrsY0hPE6pRTlU+H68/zeNdJUAIJf0LLXOm3hpTKLA19VICltl/j9VvBJpgRHdBylXEyL8HokYpjkQQk1ZXj3m7Nlo8yDdg4VcljOJWC+Xh8kxRMfK5x/VRVsYKCQXN5QlzKeqf7USRDUS/7mFoPUBW+d4kwKtGxRsWuIL2yeqzifZUTOgsh9+ZWAWxWffQZ your_email@example.com" > ~/.ssh/id_rsa.pub
 - echo -e "-----BEGIN RSA PRIVATE KEY-----\nMIIEpAIBAAKCAQEA5E3IF5x7qkdIG8HoJ6/Wcc+IU5I41f0qYCTTyc+5qPxyjW8d\nK6o49T9MhkdOd0fRkhRYptrreDSvQzz1JEHcrnMPg6C5GERyZpSeATWvTUwg9LJf\nnklztvevZGaIwmEjoG5pAicfJnYE8Ic3YBP44Qa7GNITxOqUU5VPh+vP83jXSVAC\nCX9Cy1zpt4aUyiwNfVSApbZf4/VbwSaYER3QcpVxMi/B6JGKY5EEJNWV495uzZaP\nMg3YOFXJYziVgvl4fJMUTHyucf1UVbGCgkFzeUJcynqn+1EkQ1Ev+5haD1AVvneJ\nMCrRsUbFriC9snqs4n2VEzoLIffmVgFsVn30GQIDAQABAoIBAQDPQm2sQbti0mN8\nD4Uawl8D40v30n8WhUa7EbPTOmlqKAQ2sfDhex9KRbTLEmEBmImA/Eee8o9iCTIy\n8Fv8Fm6pUHt9G6Pti/XvemwW3Q3QNpSUkHqN0FDkgecQVqVBEb6uHo3mDm4RFINX\neOmkp30BjIK9/blEw1D0sFALLOEUPaDdPMwiXtFgqfrFSgpDET3TvQIwZ2LxxTm0\ncNmP3sCSlZHJNkZI4hBEWaaXR+V5/+C1qblDCo5blAWTcX3UzqrwUUJgFi6VnBuh\n7S9Q6+CEIU+4JRyWQNmY8YgZFaAp6IOr/kyfPxTP1+UEVVgcLn3WDYwfG9og0tmz\nfzlruAgBAoGBAPfz73Pey86tNZEanhJhbX8gVjzy2hvyhT0paHg0q/H6c1VWOtUH\nOwZ3Ns2xAZqJhlDqCHnQYSCZDly042U/theP4N8zo1APb4Yg4qdmXF9QE1+2M03r\nkS6138gU/CSCLf8pCYa6pA/GmsaXxloeJGLvT4fzOZRsVav80/92XHRhAoGBAOu2\nmKh4Gr1EjgN9QNbk9cQTSFDtlBEqO/0pTepvL73UvNp/BAn4iYZFU4WnklFVBSWc\nL84Sc732xU12TAbTTUsa6E7W29pS8u7zVTxlIdQIIU5pzDyU1pNNk2kpxzte5p3Y\nPDtniPFsoYLWoH0LpsKL93t2pLAj+IOkE6f3XBq5AoGAIKaYo5N1FxQr952frx/x\nQUpK0N/R5Ng8v18SiLG26rhmM5iVSrQXC7TrHI7wfR8a9tC6qP/NqnM9NuwC/bQ0\nEEo7/GhaWxKNRwZRkmWiSFLNGk9t1hbtGU+N1lUdFtmloPIQdRNiw0kN3JTj474Q\nYI7O1EItFORnK6yxZfR6HEECgYEA1CT7MGUoa8APsMRCXyaiq15Pb8bjxK8mXquW\nHLEFXuzhLCW1FORDoj0y9s/iuKC0iS0ROX8R/J7k5NrbgikbH8WP36UxKkYNr1IC\nHOFImPTYRSKjVsL+fIUNb1DSp3S6SsYbL7v3XJJQqtlQiDq8U8x1aQFXJ9C4EoLR\nzhKrKsECgYBtU/TSF/TATZY5XtrN9O+HX1Fbz70Ci8XgvioheVI2fezOcXPRzDcC\nOYPaCMNKA5E8gHdg4s0TN7uDvKTJ+KhSg2V7gZ39A28dHrJaRX7Nz4k6t2uEBjX9\na1JidpAIbJ+3w7+hj6L299tVZvS+Y/6Dz/uuEQGXfJg/l/5CCvQPsA==\n-----END RSA PRIVATE KEY-----" > ~/.ssh/id_rsa
 - chmod 600 ~/.ssh/id_rsa*
 - eval `ssh-agent -s`
 - ssh-add ~/.ssh/id_rsa

You might might even use this trick to sneak private repo builds into public travis … but that would be evil …