This is an alternative to a proposal that I'm sure was offered with good intentions, but that I and several others shuddered at the sight of:
// Oh, the horror!
#include <https://github.com/someone/something/path/to/some/header.h>
Instead, I propose letting the first component of a /
-separated
#include <.../...>
(or similar for module import
s) be a "tag" that can be
specified in a project-wide tags.yaml
file:
#include <boost/some/boost/header.h> // tag = boost
#include <catch2/some/catch/header.h> // tag = catch2
// If the would-be tag isn't declared, it would be found as it
// is today with the normal system paths.
#include <linux/futex.h>
// no / so don't use tags
#include <map>
// don't use tags with "" includes
#include "no/tags/here.h"
(Assume all names and even the format are strawmen for now. We can bikeshed those details later, once we agree on the overall concept of prefix-tag-replacement)
# tags.yml
catch2:
repo: https://github.com/catchorg/Catch2
subdir: include/
hash: deadbeef
boost:
repo: http://github.com/some/url
hash: badf00d
benchmark:
path: third_party/benchmark/
# If path is present, remaining fields are informational only.
# They could be used by tools to simplify upgrading of vendored deps.
repo: http://github.com/google/benchmark
hash: d00dface
The top-level project can provide overrides to force child projects to use specific dependencies, or local checkouts. In general, only exactly matching git hashes are ever used. If two children in the same project specify different hashes for the same repo, the top-level project must use an override to to force a common version. Note that there is only a two-level hierarchy, the top-level project, and all dependencies.
Overrides can also be used by distros that demand that builds be done using the system version of libraries rather than vendored versions. Tools should accept an additional override file provided by the invoking user to let them overrule the project's defaults.
# These overrride everything using this tag.
GLOBAL:
boost:
# use the system version
path: /usr/include/
nifty_lib:
# use my working copy since I'm making changes to both
path: ~/develop/nifty_lib/
benchmark:
# Use a fork rather than the mainline branch
repo: http://github.com/hooli/benchmark
hash: feedf00d
# You can also override for a single project
silly_project:
boost_1_56:
use_tag: boost # replacement by reference
catch2:
# Just force it to use a different hash
repo: https://github.com/catchorg/Catch2
subdir: include/
hash: deadc0de
The C++ standard already allows
a lot of implementation flexibility in how it interprets #include
paths, so
no change to the core standard is needed to allow for this.
This is almost achievable with just build-system work today using fake include paths that just hold symlinks to the real projects. The only part that would require compiler changes is to make the include paths configurable based on where the including file is located, rather than using the same paths for the whole TU. But since that should only be needed in the (hopefully) rare case of two projects using the same tag for unrelated dependencies, this should be usable for most (sane) projects today.
- It doesn't put github links in every file!
- See 1!
- Provides a single place to specify (and change!) which version should be used for each dependency
- Makes it easy to do local development on a dependency
- Git hashes don't lie (at least once they finish moving off of SHA1)
- It uses only exact versioning to always get reproducible builds with the tested versions
- Easily extended to non-git repos and source archives (with hashes or signatures)
- Works even in projects that vendor in dependencies, but want to support building with other versions
- Plays nice with distros that require use of system libs
- Doesn't require any central repo (even github)
- Most people consider YAML human readable and editable
- Doesn't require boiling the ocean: any project can start using this with existing code, even before dependencies do
- It requires an extra file in each project
- It requires top-level projects to pick compatible versions of common libs
(These can be addressed if we like the overall direction)
- How to handle projects like boost that are generally treated as multiple packages in subdirectories.
- Could require something like
<boost.math/path/to/header.h>
so the first/
separates tag from path - Could abandon that simplicity and just let tags contain
/
and match each#include
prefix against all tags
- Could require something like
- Windows
\
vs Unix/
vs modules.
- Simplest is to treat all the same inside the paths, but that depends on the outcome of 1
- Case sensitivity
- I think the best option is to do case folding before matching
- Disallows using tags that differ only in case, but is a bad idea that isn't portable anyway
- May require compiler support
- Do we even need to support the case where multiple projects use the same tag to refer to different deps
- This is probably the hardest part of this proposal
- On the other hand this almost seems inevitable in the real world as long as some packages use common names like
benchmark
- Should the format support a field like
build commands
to describe how to build the dependency, or should that be a separate tool/format?- Certainly not a version 1 feature, I think