Skip to content

Instantly share code, notes, and snippets.

@lrettig
Last active September 12, 2021 11:55
Show Gist options
  • Save lrettig/935ca5c166e083fa92d1b64f0021c8c0 to your computer and use it in GitHub Desktop.
Save lrettig/935ca5c166e083fa92d1b64f0021c8c0 to your computer and use it in GitHub Desktop.
Compiling, testing, and running go-spacemesh on Mac M1

Compiling and running

If you just run make build you're likely to see errors like this:

make build
go build -o /Users/user/go-spacemesh/build/go-spacemesh -ldflags "-X main.version=v0.0.0-unreleased -X main.commit=3030d611e2faa40a5fb5af1e1eb48fa236138972 -X main.branch=opinion-map-dirty"  .
# github.com/spacemeshos/post/gpu
ld: warning: ignoring file /Users/user/go-spacemesh/build//libgpu-setup.dylib, building for macOS-arm64 but attempting to link with file built for macOS-x86_64
Undefined symbols for architecture arm64:
  "_scryptPositions", referenced from:
      __cgo_255ad663c373_Cfunc_scryptPositions in _x002.o
     (maybe you meant: __cgo_255ad663c373_Cfunc_scryptPositions)
  "_spacemesh_api_get_providers", referenced from:
      __cgo_255ad663c373_Cfunc_spacemesh_api_get_providers in _x002.o
     (maybe you meant: __cgo_255ad663c373_Cfunc_spacemesh_api_get_providers)
  "_spacemesh_api_stop_inprogress", referenced from:
      __cgo_255ad663c373_Cfunc_spacemesh_api_stop_inprogress in _x002.o
     (maybe you meant: __cgo_255ad663c373_Cfunc_spacemesh_api_stop_inprogress)
  "_stop", referenced from:
      __cgo_255ad663c373_Cfunc_stop in _x002.o
     (maybe you meant: __cgo_255ad663c373_Cfunc_stop, __cgo_255ad663c373_Cfunc_spacemesh_api_stop_inprogress )
ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [go-spacemesh] Error 2

These errors are due to the fact that we don't yet have an ARM-compatible release of the libgpu library that go-spacemesh depends on.

This part is very easy. Since golang allows cross-architecture compilation, you just need to set the GOARCH environment variable when building, e.g.:

GOARCH=amd64 make build

This will produce Intel compatible binaries:

file build/go-spacemesh
build/go-spacemesh: Mach-O 64-bit executable x86_64

That's all you have to do. You can run the Intel binaries directly and macOS will run them through Rosetta.

Testing

Running tests is a bit more complicated. This is because tests need to be run inside a test harness set up by the go binary itself. If you try to just naively run the tests, you'll get the same error as above:

> go test -timeout 0 -p 1 github.com/spacemeshos/go-spacemesh/tortoise
# github.com/spacemeshos/post/gpu
ld: warning: ignoring file /Users/user/go-spacemesh/build//libgpu-setup.dylib, building for macOS-arm64 but attempting to link with file built for macOS-x86_64
Undefined symbols for architecture arm64:
  "_scryptPositions", referenced from:
      __cgo_255ad663c373_Cfunc_scryptPositions in _x002.o
     (maybe you meant: __cgo_255ad663c373_Cfunc_scryptPositions)
  "_spacemesh_api_get_providers", referenced from:
      __cgo_255ad663c373_Cfunc_spacemesh_api_get_providers in _x002.o
     (maybe you meant: __cgo_255ad663c373_Cfunc_spacemesh_api_get_providers)
  "_spacemesh_api_stop_inprogress", referenced from:
      __cgo_255ad663c373_Cfunc_spacemesh_api_stop_inprogress in _x002.o
     (maybe you meant: __cgo_255ad663c373_Cfunc_spacemesh_api_stop_inprogress)
  "_stop", referenced from:
      __cgo_255ad663c373_Cfunc_stop in _x002.o
     (maybe you meant: __cgo_255ad663c373_Cfunc_stop, __cgo_255ad663c373_Cfunc_spacemesh_api_stop_inprogress )
ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
FAIL    github.com/spacemeshos/go-spacemesh/tortoise [build failed]
FAIL

If you instead try adding GOARCH=amd64, as above, you'll get a different error:

> GOARCH=amd64 go test -timeout 0 -p 1 github.com/spacemeshos/go-spacemesh/tortoise
# github.com/spacemeshos/post/gpu
../../go/pkg/mod/github.com/spacemeshos/post@v0.0.0-20210831040706-7255a25137a2/gpu/api.go:13:13: undefined: ComputeAPIClass
../../go/pkg/mod/github.com/spacemeshos/post@v0.0.0-20210831040706-7255a25137a2/gpu/api.go:22:14: undefined: cGetProviders
../../go/pkg/mod/github.com/spacemeshos/post@v0.0.0-20210831040706-7255a25137a2/gpu/api.go:87:14: undefined: cStopCleared
../../go/pkg/mod/github.com/spacemeshos/post@v0.0.0-20210831040706-7255a25137a2/gpu/api.go:100:47: undefined: cScryptPositions
../../go/pkg/mod/github.com/spacemeshos/post@v0.0.0-20210831040706-7255a25137a2/gpu/api.go:126:13: undefined: StopResult
../../go/pkg/mod/github.com/spacemeshos/post@v0.0.0-20210831040706-7255a25137a2/gpu/api.go:127:9: undefined: cStop
FAIL    github.com/spacemeshos/go-spacemesh/tortoise [build failed]
FAIL

I don't know the exact cause of this error, but it seems to be related to the same underlying issue (ARM support in the gpu lib). If you go one step further and try running go itself in compatibility mode, you get this error:

> GOARCH=amd64 arch -x86_64 go test -timeout 0 -p 1 github.com/spacemeshos/go-spacemesh/tortoise
arch: posix_spawnp: go: Bad CPU type in executable

This makes sense because go itself is compiled for ARM:

> file `which go`
/opt/homebrew/bin/go: Mach-O 64-bit executable arm64

So, in order to get tests working in compatibility mode, we need to go further up the stack and install and run the amd64 version of go itself. The easiest way to do this is to install the x86_64 variant of Homebrew, and then use that to install x86_64 packages:

# launch x86_64 shell
arch -x86_64 zsh  
# install x86_64 variant of brew 
arch -x86_64 /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)"
# install x86_64 variant of go
arch -x86_64 /usr/local/bin/brew install go

For more information, see this Stackoverflow post.

Once you've done this, you should be able to run tests in compatibility mode. Just make sure you're running the right variant of go:

> /usr/local/Cellar/go/1.17/libexec/bin/go test -timeout 0 -p 1 github.com/spacemeshos/go-spacemesh/tortoise

Success! 🥳

Goland

Note that you can also make the change in Goland by switching the Go SDK to use the amd_64 variant:

(see attached image)

Debugging

Sadly, this does not appear to work in compatibility mode. I'll update this document if I figure out how to make it work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment