Skip to content

Instantly share code, notes, and snippets.

@cpheps

cpheps/homebrew_blog.md Secret

Created July 7, 2022 15:12
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save cpheps/a2e988eb88924b46fc3ee43f97781a1e to your computer and use it in GitHub Desktop.
Save cpheps/a2e988eb88924b46fc3ee43f97781a1e to your computer and use it in GitHub Desktop.
Download ZIP
homebrew_blog_template
Raw
homebrew_blog.md

Creating Homebrew Formulas with GoReleaser

Starting Out

We chose to use GoReleaser with our distro of the OpenTelemetry Collector in order to simplify how we build and support many operating systems and architectures. It allows us to build targeting a matrix of GOOS and GOARCH targets as well as automate creating a wide range of deliverables. Ones we have utilized are building tarballs, nfpm packages, docker images, and Homebrew formula.

For this blog we wanted to focus on the Homebrew Taps capabilities in GoReleaser and our journey using it. Our goal was to make it easy for users to install our software on macOS so that they could easily try it out. We went with Homebrew as it’s familiar to many macOS users and would allow the user to try out our software then remove it just as easily when they were finished.

As we were starting out with setting up Homebrew in GoReleaser we found that documentation around creating a Homebrew formula in general was lacking. Also, it wasn’t easy to search for solutions when we encountered a problem. Homebrew provides a Formula Cookbook but it can be a bit confusing if you aren’t already familiar with building formulas.

We went through several iterations of our Homebrew formula as we learned more and more about the correct way to do things.

First we created a public repo that was to be our Homebrew formula. We would specify this to be where GoReleaser would send formula updates. We created https://github.com/observIQ/homebrew-observiq-otel-collector.

As we started setting up GoReleaser we originally used the caveats, install, and plist blocks of the brews section in GoReleaser to create our formula.

Caveats Block

The caveats block can be used to relay textual information to the user after a homebrew installation. We use it to

  1. Give info on how to start/stop/restart the homebrew service that is installed
  2. How to properly uninstall the entire formula
  3. And give info on where certain configuration files live

Install Block

Inside the install block, you can use the same brew shortcuts for installation used in the formula file itself. This ultimately will copy these same lines to the ruby formula file’s install block. For example, we use

  • prefix.install to copy files and directories to homebrew’s versioned formula directory
  • prefix.install_symlink to create symlinks in homebrew’s versioned formula directory
  • etc.install to copy files and directories to homebrew’s shared etc directory
  • bin.install to copy binary executable files to homebrew’s versioned formula’s “bin” directory
  • lib.install to copy library files to homebrew’s versioned formula’s “lib” directory

Service Blocks

The plist block was where we defined a plist file to allow our software to be run as a launchd service. The service block wasn’t supported in GoReleaser when we started, once it was we shifted to using that as it was easier to define for us and allowed a more brew native way of managing our service.

Our original plist block looked like the XML below:

plist: |
    <?xml version="1.0" encoding="UTF-8"?>
    <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
    <plist version="1.0">
    <dict>
        <key>Label</key>
        <string>com.observiq.collector</string>
        <key>ProgramArguments</key>
        <array>
        <string>#{bin}/observiq-otel-collector</string>
        <string>--config</string>
        <string>#{prefix}/config.yaml</string>
        </array>
        <key>RunAtLoad</key>
        <true/>
        <key>StandardErrorPath</key>
        <string>/tmp/observiq-otel-collector.log</string>
        <key>StandardOutPath</key>
        <string>/tmp/observiq-otel-collector.log</string>
    </dict>
    </plist>

Once we saw GoReleaser supported the service block we were able to simplify it to the following:

service: |
    run [opt_bin/"observiq-otel-collector", "--config", opt_prefix/"config.yaml"]
    environment_variables OIQ_OTEL_COLLECTOR_HOME: opt_prefix
    keep_alive true

We did have some troubles creating the service originally as there were some “magic” words that correspond to special directories in a brew installation. The cookbook documentation used these magic words in examples but did not list them like it does in the install section. We had to search the brew source code to find a list of the support “magic” words.

Here are a few of the common ones we used:

Path Variable Path
opt_prefix $HOMEBREW_PREFIX/opt/formula_name
opt_bin opt_prefix/bin
opt_include opt_prefix/include
opt_lib opt_prefix/lib

Initial Config

Here is the brews block we originally generated that created a working formula for us.

brews:
  - name: observiq-otel-collector
    tap:
      owner: observIQ
      name: homebrew-observiq-otel-collector
      branch: main
    download_strategy: CurlDownloadStrategy
    folder: Formula
    url_template: https://github.com/observIQ/observiq-otel-collector/releases/download/{{ .Tag }}/{{ .ArtifactName }}
    commit_author:
      name: observiq
      email: support@observiq.com
    homepage: "https://github.com/observIQ/observiq-otel-collector"
    license: "Apache 2.0"
    caveats: |
      ****************************************************************
      The configuration file that is run by the service is located at #{prefix}/config.yaml.
      If you are configuring a logreceiver the plugin directory is located at #{prefix}/plugins.
      ****************************************************************
      ****************************************************************
      Below are services commands to run to manage the collector service.
      If you wish to run the collector at boot prefix these commands with sudo
      otherwise the service will run at login.
      To start the collector service run:
        brew services start observiq/observiq-otel-collector/observiq-otel-collector
      To stop the collector service run:
        brew services stop observiq/observiq-otel-collector/observiq-otel-collector
      To restart the collector service run:
        brew services restart observiq/observiq-otel-collector/observiq-otel-collector
      ****************************************************************
      ****************************************************************
      To uninstall the collector and its dependencies run the following commands:
          brew services stop observiq/observiq-otel-collector/observiq-otel-collector
          brew uninstall observiq/observiq-otel-collector/observiq-otel-collector
          launchctl remove com.observiq.collector
          # If you moved the opentelemetry-java-contrib-jmx-metrics.jar
          sudo rm /opt/opentelemetry-java-contrib-jmx-metrics.jar
      ****************************************************************
    install: |
      bin.install "observiq-otel-collector"
      prefix.install "LICENSE", "config.yaml"
      prefix.install Dir["plugins"]
      lib.install "opentelemetry-java-contrib-jmx-metrics.jar"
    service: |
      run [opt_bin/"observiq-otel-collector", "--config", opt_prefix/"config.yaml"]
      environment_variables OIQ_OTEL_COLLECTOR_HOME: opt_prefix
      keep_alive true

Versioning Brew Formulas

One issue we eventually stumbled upon was versioning our software releases with Homebrew. We found after every release GoReleaser would update the Formula repo by overwriting the previous formula. A user could update the formula and run brew upgrade to easily get the latest version. The issue we ran into was what if you wanted a specific version of the Collector with a specific brew formula? You would have to know which commit in the Formula corresponds to the release you want. Not very user friendly.

This also made it hard for us to test pre-releases as we wanted GoReleaser to generate formulas for release candidates but not to overwrite the production one.

It wasn’t easy to find out how to version Homebrew Formulas. We ended up looking at the homebrew-core repo for examples of how other formulas do it.

To version a formula there are a few special things to do. The formula name needs to be of the format formula-name@major.minor.patch.rb. The added @major.minor.patch allows Homebrew to know which formula to get when specified in the brew command. Inside the formula the class name must have special formatting too. It must be of the format FormulaNameAT{Major}{Minor}{Patch}. So an example filename and corresponding class name for our Collector is observiq-otel-collector@0.6.0.rb and ObserviqOtelCollectorAT060 respectively.

That formula file will exist in the Formula directory of your repo next to the current main formula, the formula you get if you just run brew install X. You can also add a version to the main formula so users can get it by version or by the basic brew command. To do this create an Aliases directory on the same level as your Formula directory. Inside that directory create a symlink to the main formula with a versioned name.

If that’s confusing here’s the command we run to create the symlink:

cd Aliases && ln -s ../Formula/observiq-otel-collector.rb observiq-otel-collector@0.6.0

Now that we know how to create a versioned formula we need to update our GoReleaser config to generate versioned formulas for us. This should be simple since the formula and class names are taken from the name field under the brews block. We changed our name to observiq-otel-collector@{{ .Major }}.{{ .Minor }}.{{ .Patch }}. When we ran a test release with GoReleaser though we saw the class for the formula wasn’t quite right. GoReleaser was generating the class name as ObserviqOtelCollectorAT0_6_0. One quick pull request to GoReleaser and we’ve fixed that.

Here’s what our brews block of our GoReleaser config now looks like to support versions.

We also made some changes to the configuration of the Collector so there are some additional flags and files in the install and service blocks.

brews:
  - name: observiq-otel-collector@{{ .Major }}.{{ .Minor }}.{{ .Patch }}
    description: "observIQ's distribution of the OpenTelemetry Collector"
    tap:
      owner: observIQ
      name: homebrew-observiq-otel-collector
      branch: main
    download_strategy: CurlDownloadStrategy
    folder: Formula
    url_template: https://github.com/observIQ/observiq-otel-collector/releases/download/{{ .Tag }}/{{ .ArtifactName }}
    commit_author:
      name: observiq
      email: support@observiq.com
    homepage: "https://github.com/observIQ/observiq-otel-collector"
    license: "Apache 2.0"
    caveats: |
      ****************************************************************
      The configuration file that is run by the service is located at #{prefix}/config.yaml.
      If you are configuring a logreceiver the plugin directory is located at #{prefix}/plugins.
      ****************************************************************
      ****************************************************************
      Below are services commands to run to manage the collector service.
      If you wish to run the collector at boot prefix these commands with sudo
      otherwise the service will run at login.
      To start the collector service run:
        brew services start observiq/observiq-otel-collector/observiq-otel-collector@{{ .Major }}.{{ .Minor }}.{{ .Patch }}
      To stop the collector service run:
        brew services stop observiq/observiq-otel-collector/observiq-otel-collector@{{ .Major }}.{{ .Minor }}.{{ .Patch }}
      To restart the collector service run:
        brew services restart observiq/observiq-otel-collector/observiq-otel-collector@{{ .Major }}.{{ .Minor }}.{{ .Patch }}
      ****************************************************************
      ****************************************************************
      To uninstall the collector and its dependencies run the following commands:
          brew services stop observiq/observiq-otel-collector/observiq-otel-collector@{{ .Major }}.{{ .Minor }}.{{ .Patch }}
          brew uninstall observiq/observiq-otel-collector/observiq-otel-collector@{{ .Major }}.{{ .Minor }}.{{ .Patch }}
          # If you moved the opentelemetry-java-contrib-jmx-metrics.jar
          sudo rm /opt/opentelemetry-java-contrib-jmx-metrics.jar
      ****************************************************************
    install: |
      bin.install "observiq-otel-collector"
      prefix.install "LICENSE", "config.yaml", "VERSION.txt", "logging.yaml"
      prefix.install Dir["plugins"]
      lib.install "opentelemetry-java-contrib-jmx-metrics.jar"
    service: |
      run [opt_bin/"observiq-otel-collector", "--config", opt_prefix/"config.yaml", "--logging", opt_prefix/"logging.yaml", "--manager", opt_prefix/"manager.yaml"]
      environment_variables OIQ_OTEL_COLLECTOR_HOME: opt_prefix
      keep_alive true

Persisting Configuration Files

Initially in our install block of the GoReleaser config, we were using prefix.install in order to place our configuration file in the main install directory of our formula.

install: |
    …
    prefix.install "LICENSE", "VERSION.txt", "config.yaml"
    …

We found that after reinstalling this formula, our configuration file would be replaced with fresh defaults and any user changes would be lost. This wasn’t ideal, so we had to figure out how to ensure that this file would be persisted between installations.

Ultimately, the solution was to make use of Homebrew’s etc directory. This is a shared directory amongst all formulas, so we had to make an extra effort that our configuration file would be uniquely named. Now our GoReleaser install block looks something like this:

install: |
    …
    prefix.install "LICENSE", "VERSION.txt"
    etc.install "config.yaml" => "observiq_config.yaml"
    …

The problem was almost solved at this point, but we still had a preference to have this configuration file “exist” in the base formula install directory. We also preferred to have this configuration file have its original name without the “observiq_” prefix. Luckily using a symlink was a simple solution to this. Our final install block related to the configuration looked similar to this:

install: |
    …
    prefix.install "LICENSE", "VERSION.txt"
    etc.install "config.yaml" => "observiq_config.yaml"
    prefix.install_symlink etc/"observiq_config.yaml" => "config.yaml"

With this solution, our configuration lived safely in Homebrew’s etc directory with a special prefix and where it would never be automatically overwritten. At the same time, it would appear to also exist in the base installation directory without any naming prefix.

There is one more thing to note here. When there is a new installation on top of our formula, homebrew automatically adds a new version of our configuration file to its etc directory. In our case the file is named something like observiq_config.yaml.default. This will contain a clean config with default settings. This is built in behavior by Homebrew, and we haven't found any way to change this.

Conclusion

GoReleaser provides a great way to distribute your Go program via Homebrew. It allows you to focus on the installation part of your application while taking care of of all the formatting and setup of your formula file. Hopefully we've given some good insight into the pitfalls we encountered to simplify using GoReleaser and Homebrew.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment