Skip to content

Instantly share code, notes, and snippets.

@djspiewak
Created November 3, 2020 22:00
Show Gist options
  • Star 4 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save djspiewak/b8f2b4547951442102488964bc351cf9 to your computer and use it in GitHub Desktop.
Save djspiewak/b8f2b4547951442102488964bc351cf9 to your computer and use it in GitHub Desktop.

Proposed Cross-Publication Guidelines

What follows is my opinion on how we should tame all of this complexity. Specifically, how can we make it as easy as possible to keep everyone's builds and releases in-sync with the latest Dotty as we approach Scala 3. This is a very complex undertaking with a lot of moving parts. I'm attempting to draw on our experience doing this for prior Scala 2 versions, as well as personal scars from previous upgrade efforts across various Scala versions. In other words, this is a bit of a "lessons learned" phrased as "please everyone do this".

Any projects I have any control over will be following these steps to the best of our ability.

1. Cross-Publish Your Latest for Two Scala 3 Milestones

Breaking upgrades are always much easier when you can break them apart into the smallest possible steps. Publishing for the previous Scala 3 release in addition to the latest one is a very easy thing to do (since your library was already building on that version!) and it eases the migration for the downstream projects by allowing them to first upgrade to the latest version of your library, and then migrate to the latest version of Dotty.

Think of this like creating a little bit of a rolling buffer in the ecosystem.

The only exception to this should be if some sort of bug in Dotty made it impossible to publish for the previous version. In that event, just publishing for the latest version is acceptable, though please understand that it may create unforeseen pain for downstream consumers.

2. Ensure A Source-Compatible-Ish Upgrade From The Previous Version

This is a bit more complicated to explain. Imagine that my library is at version 1.2.3 and I release for Dotty 0.27.0-RC1 and 3.0.0-M1. Now imagine that M2 is released, but in the meantime, I've made several breaking changes. I could release 2.0.0 against M2 and M1, but this makes the migration considerably harder for my downstream users if this is all I do.

Instead of stopping at this, I should also go back to my 1.2.x line and release version 1.2.4 against M2 and M1, likely in addition to releasing 2.0.0 against M2 and M1. Note that I would not apply this policy transitively: I wouldn't feel compelled to release 1.2.5 against M3, for example.

Here again, the idea is to create a bit of a buffer and allow users to take the upgrade in chunks. It might be easier for a downstream project to upgrade Dotty first and then upgrade to the latest version of my library, and backporting the Dotty upgrade helps with this.

3. Communicate Everything

Miscommunication between ScalaCheck and Typelevel is the only reason that Cats 2.0 was released. (long story) There was a LOT of ecosystem churn and suffering which came about only because the Cats team simply didn't know that ScalaCheck was publishing 1.13 against the Scala 2.13 milestones. We have a relatively narrow margin for error this time, and a much larger ecosystem. Communicate any and all plans as loudly and as frequently as possible.

One easy way to do this: update your project readme to explain what version(s) are published for Dotty and which Dotty(JS) versions are supported. Making sure that things are reflected in terms of git tags also helps quite a bit. Maven Central is obviously the ultimate arbiter, but its search and consistency are unreliable at best.

4. Act Quickly

If you're a maintainer of a library which is upstream of a large transitive set of libraries, you should be as hasty as possible about releasing new versions for the latest Dotty releases. There is almost nothing which is more frustrating than having to wait for some sort of upstream dependency in order to propagate the cross-publication process down the line. If you're a top-level dependency (ScalaCheck is the most infamous example of this), then you need to be very much on the ball about moving forward quickly.

If you expect that your project will have trouble upgrading to the upcoming Dotty release, then you need to be working on snapshots in preparation, and if you don't have the time, ask for help. Hands are available, but only if they know you need assistance.

5. Don't Rely on withDottyCompat

Seriously it's really not that great. I could go on and on about the technical problems of this hack, but please just treat it as a last resort. Never tell your users "just use withDottyCompat", because that will create long-term transitive problems within the ecosystem. At the very least, sbt does not deal with diamond conflicts between _2.13 and _3.0 artifacts in a particularly graceful way. Also the Dotty unpickler is not perfect, and in any case it cannot catch more complex migration issues. Any code which does not trivially cross-build between 3.0 and 2.13 is also code which will cause problems in your user's builds whenever they use the hack.

In other words, treat this like a "normal" breaking Scala update. Try to forget that withDottyCompat exists, because for most intents and purposes, it doesn't exist.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment