Skip to content

Instantly share code, notes, and snippets.

@Arshitha
Last active June 18, 2022 20:16
Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save Arshitha/45026e56b71ae35446af2239f98dcb4b to your computer and use it in GitHub Desktop.
Save Arshitha/45026e56b71ae35446af2239f98dcb4b to your computer and use it in GitHub Desktop.
git-annex installation on M1 MacBook

Initial Error Message

(ohbm_22) arshitha@Personal-MacBook ~ % datalad install --recursive https://github.com/Arshitha/narps_open_pipelines.git
install(ok): /Users/arshitha/narps_open_pipelines (dataset)
[INFO   ] Installing Dataset(/Users/arshitha/narps_open_pipelines) to get /Users/arshitha/narps_open_pipelines recursively
[WARNING] Received an exception MissingExternalDependency(No working git-annex installation of version >= 8.20200309. Visit http://handbook.datalad.org/r.html?install for instructions on how to install DataLad and git-annex.). Canceling not-yet running jobs and waiting for completion of running. You can force earlier forceful exit by Ctrl-C.
[INFO   ] Canceled 0 out of 0 jobs. 0 left running.
[ERROR  ] No working git-annex installation of version >= 8.20200309. Visit http://handbook.datalad.org/r.html?install for instructions on how to install DataLad and git-annex. (MissingExternalDependency)
Installing: 0.00 datasets [00:10, ? datasets/s]%

Issue: Installing git-annex on M1 MacBook Air

git-annex, a dependency of DataLad, is not yet available for M1 MacBooks via homebrew package manager. Even though there are workarounds mentioned elsewhere on the internet, it's not the most comprehensive. Or at least, it didn't resolve the issue for me. But here's what worked for me:

Step 1: Install Rosetta 2

Rosetta 2 enables applications built for intel processers to work on M1/Apple Silicon MacBooks. Open up a terminal window and run the following command to install rosetta 2 softwareupdate --install-rosetta --agree-to-license

Step 2: Change settings in order to open new terminal sessions using Rosetta 2

  • Open Applications in Finder

Screenshot 2022-06-19 at 1 00 57 AM

  • Right Click on "Terminal" App -> Click on "Get Info"

Screenshot 2022-06-19 at 1 01 26 AM

  • Check "Open using Rosetta" box

Screenshot 2022-06-19 at 1 10 17 AM

Step 3: Using homebrew, try homebrew installation of git-annex with --build-from-source flag

Open a new terminal session, and run the command brew install --build-from-source git-annex. If this works, great! You are done! However, here's the error message I received when I tried the command.

(ohbm_22) arshitha@Personal-MacBook % brew install --build-from-source git-annex
Running `brew update --auto-update`...
==> Auto-updated Homebrew!
Updated 2 taps (homebrew/core and homebrew/cask).
==> New Formulae
adamstark-audiofile

You have 8 outdated formulae installed.
You can upgrade them with brew upgrade
or list them with brew outdated.

Error: Cannot install under Rosetta 2 in ARM default prefix (/opt/homebrew)!
To rerun under ARM use:
    arch -arm64 brew install ...
To install under x86_64, install Homebrew into /usr/local.

Step 4: Re-install Homebrew into /usr/local

I decided to re-install Homebrew into /usr/local since the other suggestions in the error message didn't work. To do this, I followed this excellent post/answer with all the necessary commands: https://superuser.com/a/1336026/1702689 . For quick reference, here's a list of commands mentioned in the referenced post but I would highly recommend reading the original post before running these commands.

sudo mkdir /usr/local/opt
sudo chown brodybits:staff /usr/local/opt
ls -al /usr/local # should show you the sub-directory created
cd /usr/local/opt
git clone https://github.com/Homebrew/brew

Add this Homebrew installation to PATH

export PATH=/usr/local/opt/brew/bin:$PATH

You could also append this to your .zshrc file so that Homebrew on /usr/local/opt/brew/bin/brew is used for every brew command.

Step 5: Retry Step 3

PLEASE NOTE: This step takes over an hour to run successfully so don't be surprised if it seems stuck on a particular process.

This time the command brew install --build-from-source git-annex will start running as expected! Here's a part of the standard output from my terminal for reference.

(ohbm_22) arshitha@Personal-MacBook ~ % brew install --build-from-source git-annex
==> Downloading https://ghcr.io/v2/homebrew/portable-ruby/portable-ruby/blobs/sha256:cf9137b1da5568d4949f71161a69b101f60ddb765e94d2b423c9801b67a1cb43
Already downloaded: /Users/arshitha/Library/Caches/Homebrew/portable-ruby-2.6.8_1.arm64_big_sur.bottle.tar.gz
==> Pouring portable-ruby-2.6.8_1.arm64_big_sur.bottle.tar.gz
==> Tapping homebrew/core
Cloning into '/usr/local/opt/brew/Library/Taps/homebrew/homebrew-core'...
remote: Enumerating objects: 1211558, done.
remote: Counting objects: 100% (213/213), done.
remote: Compressing objects: 100% (95/95), done.
remote: Total 1211558 (delta 130), reused 197 (delta 118), pack-reused 1211345
Receiving objects: 100% (1211558/1211558), 491.41 MiB | 10.57 MiB/s, done.
Resolving deltas: 100% (835030/835030), done.
Tapped 3 commands and 6123 formulae (6,464 files, 538.4MB).
Warning: Building pkg-config from source:
The bottle needs to be installed into /usr/local.

==> Downloading https://pkgconfig.freedesktop.org/releases/pkg-config-0.29.2.tar.gz
######################################################################## 100.0%
==> Downloading https://ghcr.io/v2/homebrew/core/gdbm/manifests/1.23
Already downloaded: /Users/arshitha/Library/Caches/Homebrew/downloads/ce78100f130177bffd255fda9df551477edd224a5080cb86cac12818954a7438--gdbm-1.23.bottle_manifest.json
==> Downloading https://ghcr.io/v2/homebrew/core/gdbm/blobs/sha256:d52ed8dbb258f11b14eb10494aeb8a2dab91c3626b11e37d8197d2fb183c489b
==> Downloading from https://pkg-containers.githubusercontent.com/ghcr1/blobs/sha256:d52ed8dbb258f11b14eb10494aeb8a2dab91c3626b11e37d8197d2fb183c489b?se=2022-06-18T15%3A25%3A00Z&sig=aV8iXUtc9v%2BM0ZZNxGrf
######################################################################## 100.0%
==> Downloading https://ghcr.io/v2/homebrew/core/mpdecimal/manifests/2.5.1
Already downloaded: /Users/arshitha/Library/Caches/Homebrew/downloads/f367c2ee08c56b88be0662703a8e4275f8657608a268c8c44e845154b0cea543--mpdecimal-2.5.1.bottle_manifest.json
==> Downloading https://ghcr.io/v2/homebrew/core/mpdecimal/blobs/sha256:255b6226cdcfaf0d40167012593e863e73dfed2884c10e7fc3eb4018e81712df
==> Downloading from https://pkg-containers.githubusercontent.com/ghcr1/blobs/sha256:255b6226cdcfaf0d40167012593e863e73dfed2884c10e7fc3eb4018e81712df?se=2022-06-18T15%3A25%3A00Z&sig=e%2Bvdv1E9nsBPoiIlOQy8
######################################################################## 100.0%
==> Downloading https://ghcr.io/v2/homebrew/core/ca-certificates/manifests/2022-04-26
Already downloaded: /Users/arshitha/Library/Caches/Homebrew/downloads/21a4c3d0a8e54b4921528b9379605fac4a683424a7a4cb0a2b5f098e18f3e229--ca-certificates-2022-04-26.bottle_manifest.json
==> Downloading https://ghcr.io/v2/homebrew/core/ca-certificates/blobs/sha256:c05a44feba2a630de2e1cefba90d3aa3f74e4d57146c0117858f648c419abeae
Already downloaded: /Users/arshitha/Library/Caches/Homebrew/downloads/52841b405d320473ad2cd55a4a6b2599d57c847d0adf58be89b05929f3c2736c--ca-certificates--2022-04-26.all.bottle.tar.gz
Warning: Building openssl@1.1 from source as the bottle needs:
- HOMEBREW_CELLAR: /usr/local/Cellar (yours is /usr/local/opt/brew/Cellar)
- HOMEBREW_PREFIX: /usr/local (yours is /usr/local/opt/brew)
  ==> Downloading https://github.com/openssl/openssl/commit/73db5d82489b3ec09ccc772dfcee14fef0e8e908.patch?full_index=1

Finally, the last part of the standard output and the good news

==> Installing git-annex
==> cabal v2-update
==> cabal v2-install --jobs=8 --max-backjumps=100000 --install-method=copy --installdir=/usr/local/opt/brew/Cellar/git-annex/10.20220525/bin --flags=+S3
==> Caveats
To restart git-annex after an upgrade:
  brew services restart git-annex
Or, if you don't want/need a background service you can just run:
  /usr/local/opt/brew/opt/git-annex/bin/git-annex assistant --autostart
==> Summary
🍺  /usr/local/opt/brew/Cellar/git-annex/10.20220525: 10 files, 60.7MB, built in 25 minutes 43 seconds
==> Running `brew cleanup git-annex`...
Disable this behaviour by setting HOMEBREW_NO_INSTALL_CLEANUP.
Hide these hints with HOMEBREW_NO_ENV_HINTS (see `man brew`).
==> Caveats
==> git-annex
To restart git-annex after an upgrade:
  brew services restart git-annex
Or, if you don't want/need a background service you can just run:
  /usr/local/opt/brew/opt/git-annex/bin/git-annex assistant --autostart

DataLad commands now work seamlessly!

References:

  1. git-annex-on-m1-mac step-by-step instructions
  2. Installation options mentioned on the official git-annex website
  3. git-annex formula page on Homebrew
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment