Skip to content

Instantly share code, notes, and snippets.

Last active November 7, 2018 23:25
Show Gist options
  • Save myitcv/79c3f12372e13b0cbbdf0411c8c46fd5 to your computer and use it in GitHub Desktop.
Save myitcv/79c3f12372e13b0cbbdf0411c8c46fd5 to your computer and use it in GitHub Desktop.
Experience report for creating a submodule within an existing module


GopherJS is a compiler from Go to JavaScript that targets browser VMs.

The package is the compiler program, responsible for transpiling Go code to JavaScript.

The package provides an API for interacting with native JavaScript.

Because of the limited runtime environment afforded by a JavaScript VM (especially in the browser), GopherJS provides special implementations of certain core/standard library pacakages. As such it is closely linked to a given Go release (ignoring point releases). Therefore, a new version of the compiler ( is released for every Go release. The implementation of (and dependent "internal" packages) is in effect tied to a given release series, i.e. 1.10, 1.10.1.... then a new release for the 1.11 series. Point releases of GopherJS therefore correspond to bug fixes in GopherJS itself. The current policy is for the GopherJS version to "closely" follow the Go version, although these releases are not tagged in the GopherJS repository.

The JavaScript interop package is, by contrast, not a function of Go release. Instead it is a function of the JavaScript language itself (GopherJS targets ECMAScript 5). As such, it does not change at all frequently, certainly not with each Go version.

Users of GopherJS broadly fall into two categories: people writing applications for browsers, and library authors writing packages to make writing such applications easier. Library authors invariably need to interact with JavaScript APIs, for things like DOM manipulation, wrapping of existing JavaScript libraries (referred to as GopherJS bindings). Those library authors often, therefore, import the package, e.g. the canonical DOM package

Both groups of users typically want to ensure their code works with the last two Go releases. Given the aforementioned implementation constraint on GopherJS itself, this requires them to depend on two versions of GopherJS. In the world of Go modules, this would translate to them relying on two major versions of

But such a major version policy within Go modules world would mean that any importers of (which is a subpackage of would be forced to also follow the same policy; a new major version for each Go release.

Hence it seems to make sense to separate into its own module; principally so that it can be versioned independently of

Given is currently a subpackage of, this involves creating a submodule.

One point of note is that depends on for:

  • implementation reasons
  • testing

But, unsurprisingly, depends on for testing.

Hence we have a cyclic module dependency:
     ^                +
     |                |
     +                v

Another critical point here is that all of this work is happening in a fork of, specifically Go 1.11 support is being added in the branch.

The goal

Given that background, our goal was therefore to create as a submodule of Furthermore, we want to version as for the upcoming Go 1.11 release. All of this happening with the fork of the project, with changes ultimately being "merged" into the branch.

The process is, as the import path suggests, hosted on Github. So the description of the process below necessarily uses terminology like Pull Request (PR) to represent the equivalent of a Gerrit CL.

PR myitcv/gopherjs#21 captures all of the commits created in this process. Each commit is also part of a separate PR.

The 5 commits are roughly summarised as follows:

  1. create submodule; this breaks the existing CI build
  2. use the submodule from
  3. use to test
  4. move to v11
  5. use to test

Each commit was maintained on a separate local branch. Where inter-branch references were required, branch names were used in go.mod as version specifications in module definitions.


Overall, the process worked very well. Unfortunately, however, the last step failed.

In the final step, we want to require from the module. But VCS does not know anything about v11, only does. Despite there being an appropriate replace directive:

require v11.0.0-20180628210949-0892b62f0d9f

replace ( => introduce_v11 => introduce_v11

per golang/go#26241 the go tool tried to resolve before examining the replace directive, and hence failed:

go: missing and .../v11/go.mod at revision 0892b62f0d9f

Per golang/go#26241, one option here would be to obviate the requirement for a require directive in the case a corresponding (versionless) replace directive exists. This would also obviate the need for any sort of "fake" version in the require directive.

There was one major pain point. In the process of creating these commits, code often needed to move between commits as fixes/changes got pushed "up" the chain to ensure that changes in a given commit were logically related to the stage in the migration. This required a number of rebases against the previous step throughout the chain. As discussed above, inter-branch references were used in go.mod definitions in both require and replace directives. However, the go command erases these branch references, replacing them with a pseudo version corresponding to the HEAD commit on the named branch at that point in time. I therefore added comments with the original branch name above the corresponding directive to act as an aide memoire. But after each rebase I was still required to replace the pseudo-version with the branch name (from the comment) in order that the go command picked up the new HEAD commit for a new pseudo-version.

Minor pain points:

There were a number of positive points from the process:

  • Separate import paths for major versions; allows module-aware Go code to depend on different GopherJS major versions in order to test against different Go releases
  • The ability to set GOFLAGS="-mod=readonly" to ensure CI did not add any unexpected dependencies; a good failsafe
  • The updates to go/build to support module-aware code. Whilst things generally feel a bit slower, it made the migration trivial (GopherJS uses the go/build package all over the shop)

Open questions

  • Whether it is necessary to have separate commits for each step described above
  • Assuming separate commits are required, whether it makes sense to have separate PRs for each commit
Copy link

bcmills commented Sep 13, 2018

Whether it is necessary to have separate commits for each step described above

I would do it all in one commit. There is a unit-test (mod_get_moved) that covers exactly this sort of cyclic split operation.

Copy link

bcmills commented Sep 13, 2018

(See example.com_split*.txt in testdata/mod for the module definitions relevant to that test.)

Copy link

jeanbza commented Nov 7, 2018

Thanks for writing this up, Paul!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment