Heroku custom compiled library .bundle/config

buildpacks.md

Buildpacks

Heroku uses buildpacks to compile your application into a slug that is used across dynos for scaling horizontally quickly. A slug is a tar.gz archive of your app’s repository with certain pre-deploy features baked into the filesystem. Since everything to run your application is included in the archive, scaling becomes a simple matter of transferring it to a dyno, unpacking, and running the appropriate process. This is how Heroku achieves scaling-by-moving-a-slider.

For example, the Ruby buildpack will:

• install ruby locally
• install the jvm/jruby (if you’re using it)
• install/run bundler and install your gems to Rails.root/vendor
• create your database.yml (which ends up reading from your app’s environment variables)
• install nodejs binaries (if you’re using them)
• precompile your assets (which has its own issues)

Here is what happens when you deploy to Heroku.

1. You push code to your remote Heroku repository. This is something like git@heroku.com:app-name.git.
2. Heroku’s git server notices and assigns the push to be handled by a special dyno used only for compiling slugs. A key thing to notice is that its build environment is the same as the dynos runtime environment, so you can compile binaries that are distributable between your dynos.
3. Looks at its api for your "repo url". If it doesn't have one yet, it makes a new one for your app.
4. Downloads your repo’s tar.gz to the dyno from S3 and unpacks it. This includes your git repository hosted on Heroku.
5. Runs the pre-receive hook on your repo. This is a Heroku-supplied script that sets up your build environment and PATH. (You can see this script using the repo plugin, listed below).
6. Runs slug-compiler.
7. slug-compiler checks out your app's code to a temporary directory, something like /tmp/build_xxxx
8. slug-compiler looks at your app's environment config variables for BUILDPACK_URL. (Set using heroku config:set)
9. If it finds it, it’ll try to clone it to the dyno and run its bin/detect script.
10. If it doesn’t find it, it runs through the Heroku standard buildpacks, running bin/detect on them until one answers positively that it can build on that codebase. (Exits status 0)
11. Assuming there is a buildpack found that says it can compile, it then runs the buildpack's bin/compile script with two arguments: $1 build_dir the directory of your app's code checkout; $2 cache_dir the "cache directory", a directory that will persist between successful slug compilations.
12. bin/compile creates/changes/deletes files in build_dir and cache_dir, and exits successfully (status 0).
13. bin/release is called with build_dir, expecting some YAML output describing default process types and addons that should be installed.
14. slug-compiler packages up the build_dir into an internally-accessible slug to distribute amongst your dynos. It also packages your whole app repo, including your cache directory, and uploads it to S3 to be used the next time you deploy.
15. Internal Heroku mechanisms push the new slug to all of your dynos and restart the processes on them. Or it spins up new dynos with your code and replaces your old dynos. Since we don't work at Heroku, we can only speculate as to which it is. My bet's on the latter.

Misc notes and tips

To get access to your app’s repo as hosted with Heroku, the heroku-repo plugin lets you download it, mess with it, and re-upload it. This was invaluable for a cache-leak bug we discovered with Heroku support which was making deploys take 2 minutes longer each deploy we did. You can also run git-gc on your repo to trim its size and potentially make your deploys go faster.

You can precompile binaries into your repo using a tool called Vulcan.

If you are compiling libraries to use for native extension based Ruby gems using your own buildpack, you need to have a way to copy a .bundle/config file into your build_dir before the Heroku ruby buildpack gets a hold of it, if you're using e.g. heroku-buildpack-multi. Unfortunately there's no current way to separate production and development environments in terms of .bundle/config. We have one in our repo called .heroku_bundle/config.

In the current version of heroku-buildpack-ruby, it also stores the .bundle/config in the cache_dir and uses that one each deploy (and then never updates it), so you’ll need to clear the cache if you change the file. In our own deployment, we clear it every deploy.

Speaking of which, your .bundle/config needs to looks something like this.

Build directives in .bundle/config have the format BUNDLE_BUILD__(gem name uppercase): --with-whatever-include=/app/libprefix/usr/include --with-whatever-lib=/app/libprefix/usr/lib

You'll notice that there are references to /app, however, on the build server they don't have an /app directory yet. In our compile step, we do something like this:

rm -f $CACHE_DIR/.bundle
cp -v -R $BUILD_DIR/.heroku_bundle $BUILD_DIR/.bundle
sed -i "s,/app,$BUILD_DIR,g" $BUILD_DIR/.bundle/config

Which replaces /app with the current build_dir, where Bundler can expect your compiled library.

In Heroku production dynos though, your app lives in /app.

z.bundle_config

---
BUNDLE_BUILD__RUBY-FILEMAGIC: --with-magic-include=/app/libmagic/usr/include --with-magic-lib=/app/libmagic/usr/lib
BUNDLE_PATH: vendor
BUNDLE_DISABLE_SHARED_GEMS: '1'
BUNDLE_CACHE_ALL: true