Bundler plugins

plugins.md

Bundler Plugins

I’ve done a fair bit of research on several well known Ruby libraries and/or tools, such as:

• Capistrano
• Ruby on Rails
• Cocoapods
• Rubygems
• Sidekiq

As I’ve looked at their codebases and existing plugins’, I found that there are essentially two areas we need to nail down to achieve maximum adoption and stability.

There are several options to achieve these two goals, but we also need something that won’t be constraining when it comes to changing internals and improving on the Bundler codebase.

Discovery

From what I’ve seen, there are essentially two ways to discover existing “plugins.”

The “registry”

The first one which, in my opinion, is of no use, is through a registry (think a global bundler config.) This would be equivalent to a Gemfile dependency, and it means that Bundler would have to deal with installing and keeping track of plugins.

This would make it super fast to load them up when the tool is invoked, but in contexts where multi-user installations are concerned (ie. bundler is installed in a global gem path but plugins are installed on a local gem path) each user would have to install and maintain their list of plugins. Not sure whether this is a problem or not, but I can imagine CIs would have an issue with this.

The “lookup”

If we treat bundler plugins as regular gems (so long as they follow a standardised naming scheme), it would be really simple to discover plugins and load them up.

Here’s a code sample of what we could use to detect and activate them:

require 'benchmark/ips'

def load_manually
  Gem.path.each do |path|
    gem_path = File.join(path, 'gems')
    next unless File.exists? gem_path
    Dir["#{ gem_path }/bundler-*-plugin"].sort.reverse.each do |gem|
      gem_name = gem.split('/').last.gsub(/\-(\d\.)+\d$/, '')
    end
  end
end

def load_with_rubygems
  Gem::Specification.find_all { |x| x.name =~ /^bundler-.*-plugin$/ }
end

Benchmark.ips do |x|
  x.config(time: 5, warmup: 2)
  
  x.report('using rubygems') { load_with_rubygems }
  x.report('manually') { load_manually }
  
  x.compare!
end

That results in the following:

Calculating -------------------------------------
      using rubygems     1.760k i/100ms
            manually   811.000  i/100ms
-------------------------------------------------
      using rubygems     18.322k (± 5.1%) i/s -     91.520k
            manually      8.445k (± 2.7%) i/s -     42.983k    

Comparison:
      using rubygems:    18322.3 i/s
            manually:     8445.5 i/s - 2.17x slower    

Our own implementation of gem discovery has its benefits, as it’s not dependent on Rubygems, albeit seemingly being slower (I suspect that Rubygems memoizes the list of installed gems whereas my code always hits the disk, but that’s another detail.)

Implementing it ourselves also has the extra bonus in that it might be faster than Rubygems if my suspicion is correct, because we don’t really need to load and parse specs. However, if moving forward, the idea is to merge with Rubygems, I’m not sure I’d go with a custom approach.

Winner

The clear winner here, in my opinion, is the lookup method. Even thought it will be slower, that difference might not even be noticeable (we can optimise the hell out of it), and it means that we won’t be making assumptions as to what state the plugins would be in.

Entry point

As for the entry point, I like how Rails has “railties.” Maybe bundler could have “bundleties.” Point is, it’s an easy entry point that simply requires us to load a file we know should be there, and that adds the functionality in one of the two ways I propose on the next section.

Adding functionality

The biggest difference I’ve seen in this respect was between Sidekiq/Rails and, for instance, Cocoapods.

Sidekiq/Rails

These two, albeit providing SOME ways of hooking your functionality in, work mostly in monkey patching / extending existing classes. Although this gives plugin authors maximum freedom when it comes to implementation, it also limits us to internal design changes (see zzak`s acts as paranoid fork.)

Cocoapods

This is the nicest plugin implementation I’ve seen. In CP, they require CLAide, which is sort of the twin brother of our Thor. There, they define a very simple structure for Hooks (think of before and after filters), with which plugins can register interests (before_install, after_cleanup, etc…)

Files of relevance are:

• https://github.com/CocoaPods/CLAide/blob/master/lib/claide/command/plugin_manager.rb
• https://github.com/CocoaPods/CocoaPods/blob/de33fb21a6a0807f7ef6eb1d86265a3c576fa057/lib/cocoapods/installer.rb#L318-L328

Winner

Cocoapods, hands down. This means we’d have to be very explicit as to what we support (there would be tons of documentation needed to be written), and consistent as to what we pass. Maybe a bundler “state” of some sort and operation specific information, like so:

• before_install: list of gems that are present in gems.rb and their versions
• after_install: the same list of gems, plus a list of what was left alone, upgraded, or downgraded.

This will require some adjustment over time, but whenever we get to a nice sweet spot where everyone gets what they need, I think we’ll be fine.

The plan

You’ve mentioned in the past that you’d like to extract a lot of bundler functionality into plugins. I’d say let’s do it for 1.9. We can easily dogfood this to ourselves and in the process have a nice, consistent, sane plugin API everyone would be happy enough to use.

I'd love working on this, as long as we define a clear plan going forward (what bundler should really be and what needs to be a plugin.)