Switching User Agents while Grabbing with Mechanize

When grabbing pages, your user-agent could compromise your efforts of ‘not getting sued‘, its easy to see that the so called
‘WWW-Mechanize/0.8.5 (http://rubyforge.org/projects/mechanize/)’ browser (mechanize default user agent) may not be ‘browsing’…

So in order to stay undetected, we change our user_agent…

require 'activesupport' #for the rand part...

class MyGrabber
  def grab
    doc = Hpricot(agent.get('www.lawyer-rich-company.com').body)
    puts doc.search('#secret_info span').inner_html
  end

  def agent
    return @agent if @agent
    @agent = WWW::Mechanize.new
    @agent.user_agent_alias = WWW::Mechanize::AGENT_ALIASES.keys.rand
    #login or what not...
    @agent
  end
end

Rubygem 1.2 is Broken, Update to Rubygems 1.3 Now

1.2 is broken/has some strange bugs, just upgrade and save yourself some headaches…

sudo gem update –system is broken in 1.2 too, so lets do it by hand…

sudo gem install rubygems-update
sudo update_rubygems

Save All Empty Strings as NIL

With the clear_empty_attributes plugin, all empty string worries are gone!

Complicate queries for empty fields (WHERE field IS NULL OR field = '')
Use of unless field.blank? (opposed to only if field)
Late-detected bugs because most of the time columns were filled or '' and suddenly they are nil
Some validations do not support :allow_blank=>true
Datebases can handle NULL better & faster than '' (especially when using LIKE)

Install

script/plugin install git://github.com/grosser/clear_empty_attributes.git

Now all empty strings will be saves as NULL.

Migrate

To take care of all other/old models we have to run a migration.
Remove any blank strings/texts from your Models:

rake clear_empty_attributes:clear_all_blank_strings MODELS=User,Movie,...

No More Whitespace Worries — Use HAML for Emails!

Tired of all the erb whitespace and those <%-end-%> madness ?

Switch to HAML for emails!!

How to?

remove any leading whitespace with capture and gsub
use == for easy string replacement
use == for newlines

Example

==Hello #{@user},
Your order has been completed on #{Time.now}.
You may now access the bought items:
==
- content = capture do
  - for item in @order.items
    - if item.shipping?
      We will send it to you soon.
    - else
      please download it here:
      ==#{item.orderable} -- #{polymorphic_url(item.orderable)}
    ==

=content.gsub(/^(  )*/,'')
==
=mail_signature

MySql: DEFAULT does NOT prevent NULL

I just came across a serious bug/gotcha when using mysql.
Integer, default 0 -> set to NULL –> NULL and not 0

Remember
always set NOT_NULL and DEFAULT!