I used to have RVM installed to manage my Ruby environment and decided to move to rbenv. I did the 'rvm implode' and removed it from Homebrew. Once I installed rbenv though, bundler stopped working whenever it tried to install Nokogiri.
Anyway, I hope this is useful and saves someone else from hours of grief!
$ bundle install
...
Building nokogiri using packaged libraries.