Skip to content

Instantly share code, notes, and snippets.

@est31
Last active April 28, 2017 11:54
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save est31/701d0e34f235f35b338ccfa2379ec620 to your computer and use it in GitHub Desktop.
Save est31/701d0e34f235f35b338ccfa2379ec620 to your computer and use it in GitHub Desktop.

Extending [replace] to work on non existent versions

Summary

Extend the entries in the [replace] section to work over non existent versions, to allow for breaking changes.

Motivation

Same as in RFC. We need a way to

  • introduce a version bump in a testing manner to a codebase and
  • apply it only to a subset of that codebase

Important terms

semver version. A string made up of three integers separated by dots. Like 0.1.0.

crate source specifier. Can be either git or path.

dependency graph. For a crate to be built c, the dependency graph is a directed graph made up of crates where an edge a → b exists iff b is a dependency of a. The root of the graph is c.

Detailed design

Right now, an entry in the [replace] section looks like this:

[replace]
"rand:0.3.15" = { path = "../rand-patched" }

Its made up of "crate-name:semver-version" = <crate source specifier>.

The old system has a couple of requirements.

a) semver-version in the [replace] section must match the version inside <crate source specifier>

b) semver-version must match the exact version of the crate version that is to be replaced

Requirement b) is inherent to how the algorithm implementing [replace] works and used for the search to replace. The proposal is to drop it (while keeping requirement a)). Therefore a change to the algorithm is neccessary.

Right now, the [replace] logic runs after cargo determines the dependency graph, as a search for the crate specified by semver-version.

When cargo builds the dependency graph, there is a phase of discovery for each crate name encountered which version of it to use. In that phase, cargo takes a list of availiable versions of a crate from the regsitry. The new behaviour should make entries in [replace] modify that list, either replacing mentions of existing versions with a pointer to the replacement location, or adding versions that don't exist on the registry to that list, also with a pointer to the replacment location.

Of course the implementors are free to chose a different, equivalent algorithm. The important part is that there is a simulation to have that version be published to the official place on crates.io.

Examples (bases heavily on RFC examples section)

(given be the basic setup from the RFC).

Scenario: prepublishing a new minor version

Like in the RFC, suppose that foo needs some changes to xml-rs, but we want to check that all of Servo compiles before pushing the changes through.

First, we change Cargo.toml for foo:

[dependencies]
xml-rs = "0.9.2"

Now, for servo, we need to record the replacement, and use the fork of foo. We don't need to modify or introduce any xml-rs dependencies:

[replace]
"xml-rs:0.9.2" = { git = "https://github.com/aturon/xml-rs", branch = "0.9.2" }

[dependencies]
foo = { git = "https://github.com/aturon/foo", branch = "fix-xml" }

With this setup:

  • When compiling foo, Cargo will resolve the xml-rs dependency to 0.9.2, and retrieve the source from the specified git branch.

  • When compiling servo, Cargo will again resolve two versions of xml-rs, this time 0.9.2 and 0.8.0, and for the former it will use the source from the git branch.

Scenario: prepublishing a new major version

What happens if foo instead needs to make a breaking change to xml-rs? The workflow is identical. For foo:

[dependencies]
xml-rs = "0.10.0"

For servo:

[replace]
"xml-rs:0.10.0" = { git = "https://github.com/aturon/xml-rs", branch = "0.10.0" }

[dependencies]
foo = { git = "https://github.com/aturon/foo", branch = "bump-xml" }

However, when we compile, we'll now get three versions of xml-rs: 0.8.0, 0.9.1 (retained from the previous lockfile), and 0.10.0. Assuming that xml-rs is a public dependency used to communicate between foo and bar this will result in a compilation error, since they are using distinct versions of xml-rs. To fix that, we'll need to update bar to also use the new, 0.10.0 version of xml-rs.

(Note that a private dependency distinction would help catch this issue at the Cargo level and give a maximally informative error message).

Cargo.lock behaviour

If there is one, the version that is being replaced should be put into Cargo.lock. All versions in the [replace] section should be put into Cargo.lock. This is (should be?) consistent with how current [replace] works.

Differences from the [prepublish] RFC

  • Most importantly, its listed in the [replace] section instead of the [prepublish] section. This keeps the system simple for the users, as there is less complexity to remember.

  • The requirement to go up the entire dependency chain with [prepublish] has been removed. This makes the feature easier to use, and also partly was required for consistency reasons, as the current [replace] works top level. There is no power lost, as if you want to stage out a change of some crate over only parts of your graph you can simply bump the major version of that crate in the Cargo.toml files where you want the replacement to happen and bump the version of it in [replace] as well.

  • Multiple different version replacements for one crate are allowed the same time, due to the crate-name:semver-version syntax. This is no inherent advantage of this proposal's approach though, it can be easily retrofitted to the RFC.

  • You need to spell out the version you want to replace by. This can be seen as advantage in having a point of reference, but also as disadvantage due to verbosity. It serves consistency concerns, and whether the version can be omitted or not should be discussed for the entire [replace] section, separately.

  • There might be differences whether warnings are printed or not, see the unresolved questions section.

If a dependency listed in [prepublish] gets published, we can show an error. If an unpublished dependency in [replace] gets published, we can't do that, as we then would have a replacement which would be just as legitimate. This is a safety disadvantage of this approach against the [prepublish] proposal. In the opinion of the author, this is only a small disadvantage, and is outweighed by the advantage of not having to introduce a completely new system.

Unresolved questions

  • Should there be always a warning when [replace] gets used? Should there be only one if a version in [replace] has not been published to crates.io yet?
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment