CMake dependency flexibility

deps.md

What we want to do

Lets say we have a library that is part of a larger CMake project, and we want to seperate it out so it can be used in more projects. For the sake of convenience lets say it is already largely seperate from the parent project, i.e. in its own directory with a CMakeLists.txt, and referenced with add_subdirectory from the parent project.

In order to fully seperate it we need to:

1. Make it buildable on its own. If it has depencies, it needs to use find_package itself to find them. If it has tests they need to be part of this build as well.
2. Make it installable. Headers and targets should be installed to CMAKE_INSTALL_PREFIX.
3. Replace uses of add_subdirectory with find_package in the parent project.

Most of the steps are fairly straight forward, although not everyone might be fully familiar with the second one: What you need to know is that a file, fooConfig.cmake, describes how to find the library foo, and that find_package(foo) will look for this file. See It's time to do CMake right, for more details.

Problems arise

When you make changes to the library, because you have to build and install it again, and if you forget it, all projects using the library will use the old version, which is particularly annoying when you can't tell why a bug you just fixed is still present.

If you want both debug, release, 32-bit and 64-bit versions of the library, you have to build and install it four times. If you make changes, you have to rebuild all versions, or a bug might be fixed in debug, but not in release.

Whether the above is worth it depends on how often the library is used, how big it is, how often it changes, if you have tools that mitigate or entirely deal with the issues etc., but it is simply the price you pay for seperating a subproject into a standalone library, and frequently worth it.

There is a bigger issue: Say we want to open source the library and a program that uses it. In order for people to build the program, they need to first build the library seperately, then build the program they actually care about using. This might be fine if the program only has that one dependency, but the more it has, the more likely it becomes that users just won't bother¹. I think telling users to

git clone --recursive https://github.com/some/program
cd program
mkdir build
cd build
cmake ..
make
sudo make install

in this day and age is reasonable. Anything more, and it damn well has to be worth it.

In short: add_subdirectory is nice because the library can be built alongside the projects using it. find_package is nice, because it allows the library to be prebuilt and shared between multiple projects using it.

It would be really nice to have something that combines the two...

Why not both?

It turns out that add_subdirectory, is misleadingly named: It is not just for subdirectories: We can in fact add an external directory to our build, if we so choose. In fact, we can do this from inside a fooConfig.cmake-script.

So, inside the the foo-library's folder, we create a fooConfig.cmake-script, in some directory that will be searched by find_package (I chose lib/foo), which then simply uses add_subdirectory, to add the library to the build tree. We have to be aware of a few things:

1. We need to specify a directory based on the to the fooConfig.cmake-script's directory. CMAKE_CURRENT_LIST_DIR lets us do that.
2. When specifying an absolute path, which is what we are doing here, we need to tell add_subdirectory, which directory to build the library in. For the most part, the name of the library will suffice.
3. We need to handle multiple uses of the library, i.e. if the library has already been found, subsequent calls should do nothing. CMake provies if(NOT TARGET) for this.
4. The names of the targets need to match up with the ones used when installing. Namely, you can specify a prefix (called a namespace) when installing targets. We can add alias targets to get compatibly named targets here².

fooConfig.cmake ends up looking like this:

if(NOT TARGET foo)
    add_subdirectory("${CMAKE_CURRENT_LIST_DIR}/../.." foo)

    add_library(Foo::foo ALIAS foo) # If we install our targets with `NAMESPACE Foo::`.
endif()

Now if we just need to tell find_package where our library is, and we can seamlessly choose between using a prebuilt library, or building the library alongside the project using it.

Git submodules

Let's say we have added the library as a git submodule. If the user clones with --recursive, they shouldn't have to tell CMake where to find it. It should just work.

We can achieve this by setting the foo_DIR variable inside CMakeLists.txt in our depending project. If the directory exists:

if(EXISTS "${CMAKE_CURRENT_SOURCE_DIR}/ext/foo/CMakeLists.txt")
    set(foo_DIR "${CMAKE_CURRENT_SOURCE_DIR}/ext/foo/lib/foo")
endif()

The takeaway

The point of this, is not that external libraries are stupid, and building everything as part of the buildtree is better. The point is that sometimes you want one thing, sometimes you want another, and you have no clue what others want, and as a result:

Choice is good, and having a choice is better than not having it, and this technique lets you choose (and provides a good default, when you don't care, and just want it to work).

Footnotes

1: Prebuilt binaries are nice and all, but if users want to contribute, they need to be able to build it on their own.

2: You could potentially run into name collisions, if you had two CMake targets, with the same name, but with different prefixes/namespaces when installed. This can be dealt with, but I would caution against it unless you know it is an issue in practice, as it involves more intrusive changes to CMakeLists.txt.

Errata

1. (2019-03-23) Now uses CMAKE_CURRENT_SOURCE_DIR in Git submodules.
2. (2019-03-23) Now checks for CMakeLists.txt instead of just the directory in Git submodules.