Ruby Parallel Processing made simple

Parallel processing for ruby:

  • Speeds up processing by %{number_of_cpus} X
  • Protects data from destructive operations

Usage

data = something_big()

require 'parallel'

#runs in number_of_your_cpus
results = Parallel.map(data) do |chunk|
  expensive_computation chunk
end

#4 processes
results = Parallel.map(data, :in_processes=>4) do |chunk|
  expensive_computation chunk
end

#same with threads (no speedup through multiple cpus,
#but speedup for blocking operations)
results = Parallel.map(data, :in_threads=>4) do |chunk|
  blocking_computation chunk
end

Parallel.each(data){|chunk| ..same.. }

Go parallel !

Advertisements

One thought on “Ruby Parallel Processing made simple

  1. Nice work! I’ve been looking for something like this for a while now. I need to run the same operations on a large number of files. Hopefully this will speed things up.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s