Tag Archives: Rails

Private gem leak / attack tester

A script to run on CI to make sure that:

  • no private gems are accidentally listed on rubygems.org (rake release happily does that for you)
  • nobody is trying to attack your private gems by releasing similar named ones

This is written for secure https://github.com/geminabox/geminabox via https://github.com/zendesk/geminastrongbox and might need to be modified to fit other gem servers.

#!/usr/bin/env ruby
def sh(command)
  result = `#{command}`
  raise "FAILED #{result}" unless $?.success?
  result
end

key = ENV.fetch('PRIVATE_SERVER_KEY')
host = ENV.fetch('PRIVATE_SERVER_HOST')
private_gem_names = sh("curl -fs 'https://#{key}@#{host}/gems'")
private_gem_names = private_gem_names.scan(%r{"#{host}/gems/gems/([^"]+)"}).flatten
puts "Found #{private_gem_names.size} private gems"
puts private_gem_names.join(", ")

exposed = sh("curl -fs 'https://rubygems.org/api/v1/dependencies?gems=#{private_gem_names.join(",")}'")
exposed = Marshal.load(exposed).map { |d| d[:name] }.uniq
puts "Found #{exposed.size} of them on rubygems.org"
puts exposed.join(", ")

if exposed.sort == ["LIST KNOW DUPLICATE HERE"].sort
  puts "All good!"
else
  raise "Hacked private gems !?: #{exposed.join(", ")}"
end

Transparant redirect for jquery ajax requests in rails with status code 278

There is no perfect backwards compatible solution out there afaik, but this is getting me pretty close to where I want it to be.

  • instead of redirecting set a empty 278 response (made up status code) for xhr requests
  • tell jquery to redirect when receiving a 278 response
  • the success handlers are still executed / there is no way to stop them, but they can either just insert the blank reply or do a != '' check
// application.js
// when we get a fake redirect response, go there
// this will still execute success handlers, but hopefully the fail or are not ugly ...
$.ajaxSetup({
  statusCode: {
    278: function(_,_,response){
      // using turbolinks to not lose back functionality
      Turbolinks.visit(response.getResponseHeader('X-278-redirect'));
    }
  }
})

# some controller
redirect_to_with_xhr signup_path

# application_controller.rb
# ajax requests follow all redirects, so we have to improvise with a
# special code and header to not get placeholders replaced with full pages.
def redirect_to_with_xhr(path)
  if request.xhr?
    response.headers["X-278-redirect"] = path
    head 278
  else
    redirect_to path
  end
end

Omniauth / OAuth integration test with webmock

Took me a while to figure out all the stubs … maybe this is useful later 🙂
I’m using this to test our facebook / twitter signup flows.

def external_redirect(url)
  yield
rescue ActionController::RoutingError # goes to twitter.com/oauth/authenticate
  current_url.must_equal url
else
  raise "Missing external redirect"
end

it "signs up" do
  twitter_id = 12345
  visit "/"
  
  stub_request(:post, "https://api.twitter.com/oauth/request_token").
    to_return(body: "oauth_token=TOKEN&oauth_token_secret=SECRET&oauth_callback_confirmed=true")

  external_redirect "https://api.twitter.com/oauth/authenticate?x_auth_access_type=read&oauth_token=TOKEN" do
    click_link "Continue with Twitter"
  end

  # https://dev.twitter.com/oauth/reference/post/oauth/access_token
  stub_request(:post, "https://api.twitter.com/oauth/access_token").
    to_return(body: "oauth_token=TOKEN&oauth_token_secret=SECRET&user_id=#{twitter_id}&screen_name=twitterapi")

  # https://dev.twitter.com/rest/reference/get/account/verify_credentials
  stub_request(:get, "https://api.twitter.com/1.1/account/verify_credentials.json?include_entities=false&skip_status=true").
    to_return(body: %{{"profile_image_url_https": "image_normal.png"}})

  visit "/auth/twitter/callback"
end

Improving Autocomplete Cache Hit Rates by Looking up Parent Caches

When a user asks for all the words starting with “foobar” let’s check the cache for fooba,foob,foo,fo,f and see if any of these results already include a full set of words.
This saved us ~30% of our db queries and makes the ui more responsive.

# given a start word of foobar check caches for ["fooba", "foob", "foo", "fo", "f"]
# to see if any of the simpler queries already had a complete response
def cached_autocomplete(start)
  limit = 1000
  short = start.dup
  shorter = Array.new(start.size - 1){ short.chop!.dup }

  cached = Rails.cache.read_multi(*shorter.map { |s| "autocomplete-#{s}" })
  cached.each_value do |tags|
    if tags.size < limit # complete response / nothing is missing
      tags.select! { |t| t.start_with?(start) } # filter non-matches
      return tags
    end
  end

  Rails.cache.fetch("autocomplete-#{start}", expires_in: 15.minutes, race_condition_ttl: 1.minute) do
    ... hit the db ... limit(limit) ...
    ... prevent multiple cache refreshes with race_condition_ttl ...
  end
end

ActionMailer / Rails: No paths in my mails please

Always paying attention that mails only use urls is a bit annoying/dangerous and also means we cannot reuse partials and cannot use nice resource routes like `link_to user.name, user`

Make ActionMailer always use full urls:

# we only want urls in our emails, never paths
module OnlyAbsoluteUrls
  def url_for(*args)
    url = super
    if url.include?("://")
      url
    else
      "#{ActionMailer::Base.default_url_options.fetch(:host)}#{url}"
    end
  end
end

class ApplicationMailer < ActionMailer::Base
  helper OnlyAbsoluteUrls
end

and to make sure it works let’s verify in all mailer tests that we did not actually generated paths:

after { deliveries.map(&:body).map(&:to_s).join.wont_include '"/' }