Category Archives: Ruby

Bundler / Docker / alpine for a super small container

Took me a while to figure out how to get this running without installing ruby-dev or tools to compile native extensions, keeping my container nice and small.

Updated: using builtin packages for bundler + io-console now since they are smaller then installing manually and the logic is simpler
(apk add –update ruby ruby-io-console ruby-bundler)
… if latest bundler is needed below Dockerfile might still be useful.

FROM alpine

RUN apk add --update ruby && rm -rf /var/cache/apk/*

ENV BUNDLER_VERSION 1.12.3
RUN gem install bundler -v $BUNDLER_VERSION --no-ri --no-rdoc

# bundler wants some library that needs core extensions ... but it won't compile
RUN mkdir /usr/lib/ruby/gems/2.2.0/gems/bundler-$BUNDLER_VERSION/lib/io
RUN touch /usr/lib/ruby/gems/2.2.0/gems/bundler-$BUNDLER_VERSION/lib/io/console.rb

# bundler does not want to install as root
RUN bundle config --global silence_root_warning 1

RUN mkdir /app
WORKDIR /app

ADD Gemfile .
ADD Gemfile.lock .
RUN bundle

Testing SuckerPunch/Celluloid vs ActiveRecord Transactions

Celluloid runs in a new thread, so it runs on a new transaction. Therefore we cannot test what was done in this transaction.

If you are not using any Celluloid callbacks then a simple unthreaded baseclass can help:

class BackgroundJobTestable
  def perform
    ...
end

class BackgroundJob < BackgroundJobTestable
  include SuckerPunch::Job
  workers 2
end

An alternative way to fix this is to disable AR multithreading support via a mocking library like mocha … this will break any code that truly runs in parallel, but if you only run 1 code path at a time this should work fine.

ActiveRecord::Base.stubs(connection: ActiveRecord::Base.connection)

Automated lambda code upload to S3 with CloudFormation

Maintaining lambda code directly in CloudFormation only works with zipfile property on nodejs and even there it is limited to 2000 characters. For python and bigger lambdas, we now use this ruby script to generate the s3 object that is set in the CloudFormation template.

require 'aws-sdk-core'

def code_via_s3(file, handler)
  bucket = "my-lambda-staging-area"
  content = File.read(file)
  sha = Digest::MD5.hexdigest(content)
  key = "#{file.gsub('/', '-')}-#{sha}.zip"

  # zip up the content (gzip is not supported)
  # needs to be at bottom of zip to support inline editing
  # and match the handler name
  content = `cd #{File.dirname(file)} && zip --quiet - #{File.basename(file)}`
  raise "Zip failed" unless $?.success?

  # upload to s3 (overwriting it ... checking for existance takes the same time ...)
  c = Aws::S3::Client.new
  begin
    c.put_object(body: content, bucket: bucket, key: key)
  rescue Aws::S3::Errors::NoSuchBucket
    c.create_bucket(bucket: bucket)
    retry
  end

  {
    "Handler" => File.basename(file).sub(/\..*/, '') + '.' + handler,
    "Code" => {"S3Bucket" => bucket, "Key" => key}
  }
end

Stubbing Ruby AWS SDK XML with webmock

Stubbing the XML AWS expects is not easy (expects lists to have member keys) and has lots of repetitive elements like XyzResponse + XyzRequest … so I wanted to share a few useful helpers that make it dry.

(Alternative: use stub_requests, example PR)

  # turn ruby hashes into aws style xml
  def fake_xml(name, body={})
    xml = {&quot;#{name}Result&quot; =&gt; body}
      .to_xml(root: &quot;#{name}Response&quot;, camelize: true).
      .gsub(/ type=&quot;array&quot;/, '')
    loop do
      break unless xml.gsub!(%r{&lt;(\S+)s&gt;\s*&lt;\1&gt;(.*?)&lt;/\1&gt;\s*&lt;/\1s&gt;}m, &quot;&lt;\\1s&gt;&lt;member&gt;\\2&lt;/member&gt;&lt;/\\1s&gt;&quot;)
    end
    xml
  end

  def expect_aws_request(method, url, action, response={})
    request = stub_request(method, url).
      with(:body =&gt; /Action=#{action}(&amp;|$)/)
    request = if response.is_a?(Exception)
      request.to_raise(response)
    else
      request.to_return(:body =&gt; fake_xml(action, response))
    end
    requested &lt;&lt; request
    request
  end

  def expect_upload_certificate
    expect_aws_request(
      :post, &quot;https://iam.amazonaws.com/&quot;,
      &quot;UploadServerCertificate&quot;,
      {server_certificate_metadata: {arn: 'FAKE-ARN'}}
    )
  end

  after { requested.each { |r| assert_requested r } }

  it &quot;uploads a cert&quot; do
    expect_upload_certificate
    manager.upload.must_equal 'FAKE-ARN'
  end

Improving sparkle_formation method_missing

Sparkle formation has the habit of swallowing all typos, which makes debugging hard:

foo typo
dynanic! :bar
# ... builds
{
  "typo": {},
  "foo": "<#SparkleFormation::Struct",
  "dymanic!": {"bar": {}}
}

let’s make these fail:

  • no arguments or block
  • looks like a method (start with _ or end with !)
# calling methods without arguments or blocks smells like a method missing
::SparkleFormation::SparkleStruct.prepend(Module.new do
   def method_missing(name, *args, &block)
     caller = ::Kernel.caller.first

     called_without_args = (
       args.empty? &&
       !block &&
       caller.start_with?(File.dirname(File.dirname(__FILE__))) &&
       !caller.include?("vendor/bundle")
     )
     internal_method = (name =~ /^_|\!$/)

     if called_without_args || internal_method
       message = "undefined local variable or method `#{name}` (use block helpers if this was not a typo)"
       ::Kernel.raise NameError, message
     end
     super
   end
end)