myitcv/report.md

## report.md

      
    Raw
  

              report.md
            
          
    Background

GopherJS is a compiler from Go to JavaScript that targets browser VMs.
The package github.com/gopherjs/gopherjs is the compiler program, responsible for transpiling Go code to JavaScript.
The package github.com/gopherjs/gopherjs/js provides an API for
interacting with native JavaScript.
Because of the limited runtime environment afforded by a JavaScript VM (especially in the browser), GopherJS provides
special implementations of certain core/standard library pacakages. As such it is closely linked to a given Go release
(ignoring point releases). Therefore, a new version of the compiler (github.com/gopherjs/gopherjs) is released for
every Go release. The implementation of github.com/gopherjs/gopherjs (and dependent "internal" packages) is in effect
tied to a given release series, i.e. 1.10, 1.10.1.... then a new release for the 1.11 series. Point releases of GopherJS
therefore correspond to bug fixes in GopherJS itself. The current policy is for the GopherJS version to "closely" follow
the Go version, although these releases are not tagged in the GopherJS repository.
The JavaScript interop package github.com/gopherjs/gopherjs/js is, by contrast, not a function of Go release. Instead
it is a function of the JavaScript language itself (GopherJS targets ECMAScript 5). As such, it does not change at all
frequently, certainly not with each Go version.
Users of GopherJS broadly fall into two categories: people writing applications for browsers, and library authors
writing packages to make writing such applications easier. Library authors invariably need to interact with JavaScript
APIs, for things like DOM manipulation, wrapping of existing JavaScript libraries (referred to as GopherJS bindings).
Those library authors often, therefore, import the github.com/gopherjs/gopherjs/js package, e.g. the canonical DOM
package honnef.co/go/js/dom.
Both groups of users typically want to ensure their code works with the last two Go releases. Given the aforementioned
implementation constraint on GopherJS itself, this requires them to depend on two versions of GopherJS. In the world of
Go modules, this would translate to them relying on two major versions of github.com/gopherjs/gopherjs.
But such a major version policy within Go modules world would mean that any importers of
github.com/gopherjs/gopherjs/js (which is a subpackage of github.com/gopherjs/gopherjs) would be forced to also
follow the same policy; a new major version for each Go release.
Hence it seems to make sense to separate github.com/gopherjs/gopherjs/js into its own module; principally so that it
can be versioned independently of github.com/gopherjs/gopherjs.
Given github.com/gopherjs/gopherjs/js is currently a subpackage of github.com/gopherjs/gopherjs, this involves
creating a submodule.
One point of note is that github.com/gopherjs/gopherjs depends on github.com/gopherjs/gopherjs/js for:

implementation reasons
testing

But github.com/gopherjs/gopherjs/js, unsurprisingly, depends on github.com/gopherjs/gopherjs for testing.
Hence we have a cyclic module dependency:
github.com/gopherjs/gopherjs/js
     ^                +
     |                |
     +                v
 github.com/gopherjs/gopherjs

Another critical point here is that all of this work is happening in a fork of https://github.com/gopherjs/gopherjs,
specifically https://github.com/myitcv/gopherjs. Go 1.11 support is being added in the
https://github.com/myitcv/gopherjs/tree/go1.11 branch.
The goal

Given that background, our goal was therefore to create github.com/gopherjs/gopherjs/js as a submodule of
github.com/gopherjs/gopherjs. Furthermore, we want to version github.com/gopherjs/gopherjs as
github.com/gopherjs/gopherjs/v11 for the upcoming Go 1.11 release. All of this happening with the
https://github.com/myitcv/gopherjs fork of the github.com/gopherjs/gopherjs project, with changes ultimately being
"merged" into the https://github.com/myitcv/gopherjs/tree/go1.11 branch.
The process

github.com/gopherjs/gopherjs is, as the import path suggests, hosted on Github. So the description of the process
below necessarily uses terminology like Pull Request (PR) to represent the equivalent of a Gerrit CL.
PR myitcv/gopherjs#21 captures all of the commits created in this process. Each commit is also
part of a separate PR.
The 5 commits are roughly summarised as follows:

create github.com/gopherjs/gopherjs/js submodule; this breaks the existing CI build
use the github.com/gopherjs/gopherjs/js submodule from github.com/gopherjs/gopherjs
use github.com/gopherjs/gopherjs to test github.com/gopherjs/gopherjs/js
move github.com/gopherjs/gopherjs to v11
use github.com/gopherjs/gopherjs/v11 to test github.com/gopherjs/gopherjs/js

Each commit was maintained on a separate local branch. Where inter-branch references were required, branch names were
used in go.mod as version specifications in module definitions.
Conclusion

Overall, the process worked very well. Unfortunately, however, the last step failed.
In the final step, we want to require github.com/gopherjs/gopherjs/v11 from the github.com/gopherjs/gopherjs/js module.
But VCS https://github.com/gopherjs/gopherjs does not know anything about v11, only https://github.com/myitcv/gopherjs does.
Despite there being an appropriate replace directive:
require github.com/gopherjs/gopherjs/v11 v11.0.0-20180628210949-0892b62f0d9f

replace (
	github.com/gopherjs/gopherjs => github.com/myitcv/gopherjs introduce_v11

	github.com/gopherjs/gopherjs/js => github.com/myitcv/gopherjs/js introduce_v11
)

per golang/go#26241 the go tool tried to resolve github.com/gopherjs/gopherjs/v11 before
examining the replace directive, and hence failed:
go: github.com/gopherjs/gopherjs/v11@v11.0.0-20180628210949-0892b62f0d9f: missing github.com/gopherjs/gopherjs/go.mod and .../v11/go.mod at revision 0892b62f0d9f

Per golang/go#26241, one option here would be to obviate the requirement for a require directive in the
case a corresponding (versionless) replace directive exists. This would also obviate the need for any sort of "fake" version
in the require directive.
There was one major pain point. In the process of creating these commits, code often needed to move between commits as
fixes/changes got pushed "up" the chain to ensure that changes in a given commit were logically related to the stage in
the migration. This required a number of rebases against the previous step throughout the chain. As discussed above,
inter-branch references were used in go.mod definitions in both require and replace directives. However, the go
command erases these branch references, replacing them with a pseudo version corresponding to the HEAD commit on the
named branch at that point in time. I therefore added comments with the original branch name above the corresponding
directive to act as an aide memoire. But after each rebase I was still required to replace the pseudo-version with the
branch name (from the comment) in order that the go command picked up the new HEAD commit for a new pseudo-version.
Minor pain points:

Despite setting GO111MODULE=on I needed to move development outside of my GOPATH because of
golang/go#26046 (comment)

There were a number of positive points from the process:

Separate import paths for major versions; allows module-aware Go code to depend on different GopherJS major versions
in order to test against different Go releases
The ability to set GOFLAGS="-mod=readonly" to ensure CI did not add any unexpected dependencies; a good failsafe
The updates to go/build to support module-aware code. Whilst things generally feel a bit slower, it made the
migration trivial (GopherJS uses the go/build package all over the shop)

Open questions


Whether it is necessary to have separate commits for each step described above
Assuming separate commits are required, whether it makes sense to have separate PRs for each commit