Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Microgem for compiling assets in parallel
# -*- encoding: utf-8 -*-
Gem::Specification.new do |s|
s.name = 'parallel_assets_compiler'
s.version = '0.2.0'
s.platform = Gem::Platform::RUBY
s.author = 'Jørgen Orehøj Erichsen'
s.email = 'joe@erichsen.net'
s.summary = 'Compile assets in parallel'
s.description = 'Compile assets in parallel to speed up deployment'
s.files = ['parallel_assets_compiler.rb']
s.require_path = '.'
s.add_dependency('parallel')
s.add_dependency('actionpack')
end
# Compile assets in parallel to speed up deployment
#
# Works by monkey patching actionpack from Rails 3.2.5
#
# Use it by adding this to your Gemfile and run bundle install
#
# gem 'parallel_assets_compiler', :git => 'git://gist.github.com/2873091.git'
#
# Inspired by
# https://github.com/steel/sprockets/commit/6327afc3341e34efd1dfdcfad08e3b9d85f7fd4a
# https://github.com/hornairs/sprockets-rails-parallel
require 'sprockets/static_compiler'
require 'parallel'
module Sprockets
class StaticCompiler
def compile
# Collect paths
logical_paths = []
env.each_logical_path do |logical_path|
if File.basename(logical_path)[/[^\.]+/, 0] == 'index'
logical_path.sub!(/\/index\./, '.')
end
logical_paths << logical_path if compile_path?(logical_path)
end
# Compute!
results = Parallel.map(logical_paths) do |logical_path|
if asset = env.find_asset(logical_path)
[logical_path, write_asset(asset)]
end
end
write_manifest(Hash[results]) if @manifest
end
end
end
@brentd

This comment has been minimized.

Copy link

@brentd brentd commented Aug 3, 2012

Nice dude.@grosser did a great job on the parallel gem. I didn't know about this or the other two works you reference in the comment above.

One small issue here: when I did a test run of this, my manifest.yml was a empty hash. Since the Parallel.map block is executed within each worker fork, it'll be using the work process' memory as opposed to the parent process, so the mutations to the manifest are effectively ignored. Parallel.map is a great piece of awesome since it'll handle marshalling the return value of the block back to the parent process via pipes. See my gist for what I did: https://gist.github.com/3221581

@joerichsen

This comment has been minimized.

Copy link
Owner Author

@joerichsen joerichsen commented Aug 3, 2012

Duh, you're right about the empty manifest file - thx :-)
I've changed my version based on your suggestions and bumped the version to 0.2.0.

@brentd

This comment has been minimized.

Copy link

@brentd brentd commented Aug 3, 2012

Sweet. Nice shortcut using Hash[] - hadn't seen that before.

@phuongnd08

This comment has been minimized.

Copy link

@phuongnd08 phuongnd08 commented Aug 22, 2012

Cut my assets compilation time from 1m13secs to 53 secs on a Mac i5 Dual SSD machine.

@phuongnd08

This comment has been minimized.

Copy link

@phuongnd08 phuongnd08 commented Aug 22, 2012

And from 95.5 secs to 57.3 secs on Heroku. That's awesome. Thanks @joerichsen for the awesome gem and @brendtd for the correction

@masterkain

This comment has been minimized.

Copy link

@masterkain masterkain commented Sep 16, 2012


Undumpable Exception -- #<NoMethodError: undefined method `options=' for true:TrueClass>
/Users/kain/.rvm/gems/ruby-1.9.3-p194/gems/parallel-0.5.18/lib/parallel.rb:173:in `work_in_processes'
/Users/kain/.rvm/gems/ruby-1.9.3-p194/gems/parallel-0.5.18/lib/parallel.rb:55:in `map'

@butsjoh

This comment has been minimized.

Copy link

@butsjoh butsjoh commented Sep 26, 2012

same issue Undumpable Exception -- #<NoMethodError: undefined method `options=' for true:TrueClass>

@aaronjensen

This comment has been minimized.

Copy link

@aaronjensen aaronjensen commented Jan 18, 2013

I get this w/ sprockets 2.2.2:

undefined method `compile_path?' for #<Sprockets::StaticCompiler:0x007fa14f4fc048>
/Users/user/bundle/ruby/1.9.1/bundler/gems/2873091-2e1e123a0036/parallel_assets_compiler.rb:26:in `block in compile'
@aaronjensen

This comment has been minimized.

Copy link

@aaronjensen aaronjensen commented Jan 18, 2013

Looks like that method may have been removed from actionpack? I tried this one from turbo-sprockets-rails3:

      private
      def compile_path?(logical_path)
        paths.each do |path|
          case path
          when Regexp
            return true if path.match(logical_path)
          when Proc
            return true if path.call(logical_path)
          else
            return true if File.fnmatch(path.to_s, logical_path)
          end
        end
        false
      end

It worked fine on my dev box, but on my build agent I got the Undumpable Exception :(

@afeld

This comment has been minimized.

Copy link

@afeld afeld commented May 30, 2013

Make it a real gem!

@morganchristiansson

This comment has been minimized.

Copy link

@morganchristiansson morganchristiansson commented Mar 17, 2014

Doesn't work with compile_path? error.

Make it a real gem!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.