Go support for Meson

notes.md

mesonbuild/meson#123

Motivation

I like Go a lot, but I'm looking for a more powerful tool for organizing my projects. Here's some of the things I could hopefully do in Meson if we added support

1. build multiple executables with a varied set of params
   • for instance cross compile some
   • set flags on others
2. build nice docs
   • compile markdown to roff, or html, or etc.
3. package those executables into native packages (e.g. RPM)
4. Build and link in C code

I can do all these things, but I am copying scripts around, and so is my team at work :)

Other Things I've Used

I've built Go with Bazel. At my company we have a project that compiles C++ and Go into a single binary, and we use Bazel to do that, becuase we require integration with Envoy, and Envoy uses Bazel. It works fine, but it's way too heavy to recommend for new projects.

Getting Started

It seems that the Rust + Meson integration is some relevant prior art for me to look at. So I have some questions about that, and some general questions, too.

• The Go tool downloads dependencies on its own, and the custom is to store them a ./vendor repo. Cargo does something similar. How does Meson interact with cargo generally? What's the Meson philosophy on interacting with language-specific package managers?
• A Go project declares all its direct dependencies in a go.mod file, and the go tool builds a go.sum file with all (?) the transient dependencies as well. Would a Go plugin for Meson be expected to parse these files? How much can I delegate to the official Go tools?

I cloned the Meson repo the other day, and grepped for "Rust". I'm sure I've missed a few things, but here's some files that caught my eye

compilers/compilers.py
compilers/detect.py
compilers/rust.py
modules/unstable_rust.py
interpreter/interpreter.py
templates/rusttemplates.py

Can you provide a study guide for walking through the codebase, and/or a high level list of things one has to implement to add support for Go?

related post; https://recursiveascent.blogspot.com/2021/02/rust-in-meson-057-edition.html

it seems like there is a minimum set of functionality one can implement, and then cooler features (like first class test support) can be added later

Related post is mine :)

So, let's start with the parts I'm familiar with (I've done a bit of golang programming, but it was about 6 years ago and state of the art go has changed a lot since then). Cargo.

Meson has support for external dependencies, either through the external_project module, or the cmake module. These work in different ways, but achieve the same results. external_project if for classic autotools style dependencies with a configure script and then an invocation to make. external_project calls ./configure during meson setup, and then treats make as a build target that outputs some number of libraries that are then returned from the make call. CMake works by replaying the cmake build as a JSON log, and then generating Meson AST from that, and then interpreting that as a native meson build. The advantage of the external_project module is that it's more robust, the disadvantage is that it needs all of the external tools, and ninja cannot parallelize as well, due to not knowing what make is going to generate. The advantage of the cmake approach is that everything is a native ninja target, Ninja's DAG is more optimal, and meson can better optimize the targets generated.

I've written cargo support twice now. I have a PR (it might be closed now) and another, newer draft I've been working on from scratch because I dislike some of the approach I took with the original. I'm taking the cmake approach with cargo, because you declare the build in toml, which is a declarative file format. This makes it easy to parse the toml and generate meson AST, and we still get all of the advantages of a single meson invocation (in fact, my branch was faster at building some pure cargo projects than cargo was!). I can push the new version up to my public repo if you want to look at it, though it's still pretty early in the rewrite. I'm choosing not to invoke cargo, because it 1) doesn't really help me to do so, and 2) not requiring cargo may be helpful in low level settngs where you have vendored your rust dependencies in meson's subprojects folder.

Meson expects all vendored or fetched code to go into a single directory, usually that is subprojects.

Cython might be a better target to look at as it's fresher, and less featureful than Rust. In essence what you need is:

1. A (or possibly two) compiler representations (one for go and one for gcc-go?)
2. detection code for those compilers
3. backend code to generate rules if they're not like the C/C++ rules

I'd start with just being able to declare go code and compile with Meson first, then worry about integrating with golang's build system second.