Why C/C++ package managers fail for now ?

Notes on C&C++ package managers.md

Motivation

As for now (mid 2016), there doesn't seem to be a C/C++ package manager that stands out of the crowd. To understand the reasons behind this failure, I will try to list in this README most C/C++ package managers, highlight differences between them, then list critics that are made about them.

Note: this README is merely a gathering of personnal notes, doesn't intend to be a reference in any way.

A standard proposal : http://open-std.org/JTC1/SC22/WG21/docs/papers/2016/p0235r0.pdf

Package managers

CPM

• repository : [https://github.com/iauns/cpm]
• website : [http://www.cpm.rocks/]
• current status : dead. Last commit from Feb. 17 2015

Conan

• repository : [https://github.com/conan-io/conan]
• website : [https://conan.io/]
• current status : in development. Website uncomplete (profile page for instance). Core however seems functionnal

For being in the process of trying it, a few remarks:

• Sure, it uses python. I personnaly like python, so no big deal for me.
• However, I dislike the fact that you can write conanfile.py and conanfile.txt. It is not unified, although they might serve different purposes.

Biicode

• dead

clib

• repository : https://github.com/clibs/clib

Critics

• (Conan) Not using C/C++ as the language for developping the package manager itself
• (Conan) Lacking support for cross-compilation ?
• (Conan) Copies all files from a local module into local Conan store, which can be an issue with large libs
• (Conan) Possible to overwrite an existing version
• (Conan) Doesn't seem to support fully CI flows !

Requirements & Challenges

Dependency hell

See http://dominictarr.com/post/25516279897/why-you-should-never-write-a-package-manager

Dependency hell is for instance when a package A depends on B1.0 and C, but C depends on B2.0. In this case, putting all B module in the search PATH will generate a conflict. Need to resolve version of B that satisfies both.

Note: Doesn't seem to be a cause of failure of adoption. For instance, python doesn't have such mechanism implemented yet

Support for large packages

Large packages, ideally should not need to be downloaded for each-new projet but only each time a different version of the package is required by a project. And, in the same time, several projects might need different version of the package, with possibly different binaries (debug, release, build config in general...) To solve this, Conan uses a global install location with a different sha for each binary, to differentiate them using build configuration (debug, release, 32, 64, compiler...).

Cross-compilation

It seems the package manager needs to be thought from the ground up for cross-compilation. Arguments ? Key points for handling this issue ?

Lack of generic build process

There is not a single way to build a C++ project, but many, very diverses.

C/C++ offer many different build tools (unlike framework/languages like nodeJS, python, etc, where there is always a single build/execution program).

In C++, there is no convention either on project folder hierarchy. Depending on your build tools (Makefile, Ninja, CMAKE, etc) and project type (native app, kernel, embedded real-time firmware) there can be better and different ways to structure a project.

Also, there is no consensus either on build tools. This could mean that a package manager must choose and enforce a specific build tool and build flow, which of course never makes everyone happy.

How to solve this ?

Hosting package elsewhere than central repository

Transitive dependency handling

https://www.reddit.com/r/cpp/comments/42wme6/modern_build_systems/d1hkyni

Personnal questions

Embedded librairies

How to structure low-level embedded (bare-metal, real-time) librairies ?

• If the library is implemented at some level with hardware abstraction, that calls hardware specific routines, how should they be structured ?
• Should the hardware-specific routines be packages on their own, making it easy to port the library for anyone on a new platorm ?
• Or part of the core library itself ?

Open questions

From predatorian3

what's a good strategy for dependency management so your contributors, or maintainers can also use the same libraries that you do?

Resources

• evanjones.ca/build-c.html

There is also Vcpkg.

vcpkg is currenty in preview mode, lots of libs cannot be build. And more, it does not support clang in macOS, only gcc supported and alot of libs in macOS requires clang.

FYI npm allow packages to use C++ modules which are compiled on install (some packages sometime try to fetch a pre-built binary then fallback on compilation).
Given enough flexibility, every build system could work, and no one would care about the built system you would use as long as you did you work and it compiles when installed through the package manager.

There is also Buckaroo

Is buckaroo dead? I can't find any activity for it from 2018

@KelsonBall

Buckaroo is very much alive, with v2 released quite recently.

Some differences between Buckaroo and the alternatives:

• Modern Build System. Other systems are built around CMake (or in practice 90% CMake), whereas Buckaroo uses one modern packaging format. This is because CMake (even "modern" CMake) does not scale to a large ecosystem of dependencies, since it is not hermetically sealed and does not have a sane scripting language. Note that there are migration + wrapping tools from CMake to Buckaroo.
• Full decentralization. Buckaroo discovers dependencies directly from Git and HTTP. Note that other solutions are cook-book driven, and so are not practically decentralized, even if they claim to be.
• Reproducible Installations. Buckaroo installs packages with information only from the lock-file, so you get the exact same code, every time.

Basically, if you want an experience like Cargo / NPM / Godep then Buckaroo is the closest to that. Other solutions are less opinionated, but you lose out on simplicity.