Start a clean module
matthijs@grubby:~/foo$ go mod init foo
Just to get a clean slate, not influenced by my existing $GOPATH
matthijs@grubby:~/foo$ export GOPATH=$(pwd)/build
Add dependency to the code
matthijs@grubby:~/foo$ cat main.go
package main
import "github.com/asdine/storm"
func main() {
storm.Open("")
}
Build and see that only minimal dependencies were added to go.mod
matthijs@grubby:~/foo$ go build
go: finding github.com/asdine/storm v2.1.2+incompatible
go: downloading github.com/asdine/storm v2.1.2+incompatible
go: finding go.etcd.io/bbolt v1.3.2
go: downloading go.etcd.io/bbolt v1.3.2
matthijs@grubby:~/foo$ cat go.mod
module foo
require (
github.com/asdine/storm v2.1.2+incompatible
go.etcd.io/bbolt v1.3.2 // indirect
)
Then go mod vendor adds one more dependency for some reason:
matthijs@grubby:~/foo$ go mod vendor
go: finding golang.org/x/sys/unix latest
go: finding golang.org/x/sys latest
go: downloading golang.org/x/sys v0.0.0-20190410235845-0ad05ae3009d
matthijs@grubby:~/foo$ cat go.mod
module foo
require (
github.com/asdine/storm v2.1.2+incompatible
go.etcd.io/bbolt v1.3.2 // indirect
golang.org/x/sys v0.0.0-20190410235845-0ad05ae3009d // indirect
)
matthijs@grubby:~/foo$ find vendor -type d
vendor
vendor/golang.org
vendor/golang.org/x
vendor/golang.org/x/sys
vendor/golang.org/x/sys/unix
vendor/github.com
vendor/github.com/asdine
vendor/github.com/asdine/storm
vendor/github.com/asdine/storm/index
vendor/github.com/asdine/storm/internal
vendor/github.com/asdine/storm/q
vendor/github.com/asdine/storm/codec
vendor/github.com/asdine/storm/codec/json
vendor/go.etcd.io
vendor/go.etcd.io/bbolt
Running go list all actually adds more dependencies, while I was expecting it to only list existing dependencies:
matthijs@grubby:~/foo$ go list all
go: finding github.com/stretchr/testify/require latest
go: finding github.com/golang/protobuf/proto latest
go: finding github.com/Sereal/Sereal/Go/sereal latest
go: finding github.com/Sereal/Sereal/Go latest
go: finding github.com/Sereal/Sereal latest
go: finding gopkg.in/check.v1 latest
can't load package: package foo/build/pkg/mod/github.com/!sereal/!sereal@v0.0.0-20190409170602-963d7e218945/Go/sereal: can only use path@version syntax with 'go get'
can't load package: package foo/build/pkg/mod/github.com/!sereal/!sereal@v0.0.0-20190409170602-963d7e218945/Go/sereal/cmd/dsrl: can only use path@version syntax with 'go get'
can't load package: package foo/build/pkg/mod/github.com/!sereal/!sereal@v0.0.0-20190409170602-963d7e218945/Go/sereal/cmd/fuzzer: can only use path@version syntax with 'go get'
can't load package: package foo/build/pkg/mod/github.com/asdine/storm@v2.1.2+incompatible: can only use path@version syntax with 'go get'
can't load package: package foo/build/pkg/mod/github.com/asdine/storm@v2.1.2+incompatible/codec: can only use path@version syntax with 'go get'
can't load package: package foo/build/pkg/mod/github.com/asdine/storm@v2.1.2+incompatible/codec/gob: can only use path@version syntax with 'go get'
can't load package: package foo/build/pkg/mod/github.com/asdine/storm@v2.1.2+incompatible/codec/internal: can only use path@version syntax with 'go get'
can't load package: package foo/build/pkg/mod/github.com/asdine/storm@v2.1.2+incompatible/codec/json: can only use path@version syntax with 'go get'
can't load package: package foo/build/pkg/mod/github.com/asdine/storm@v2.1.2+incompatible/codec/msgpack: can only use path@version syntax with 'go get'
can't load package: package foo/build/pkg/mod/github.com/asdine/storm@v2.1.2+incompatible/codec/protobuf: can only use path@version syntax with 'go get'
can't load package: package foo/build/pkg/mod/github.com/asdine/storm@v2.1.2+incompatible/codec/sereal: can only use path@version syntax with 'go get'
can't load package: package foo/build/pkg/mod/github.com/asdine/storm@v2.1.2+incompatible/index: can only use path@version syntax with 'go get'
can't load package: package foo/build/pkg/mod/github.com/asdine/storm@v2.1.2+incompatible/internal: can only use path@version syntax with 'go get'
can't load package: package foo/build/pkg/mod/github.com/asdine/storm@v2.1.2+incompatible/q: can only use path@version syntax with 'go get'
can't load package: package foo/build/pkg/mod/github.com/davecgh/go-spew@v1.1.0/spew: can only use path@version syntax with 'go get'
can't load package: package foo/build/pkg/mod/github.com/pmezard/go-difflib@v1.0.0/difflib: can only use path@version syntax with 'go get'
can't load package: package foo/build/pkg/mod/github.com/vmihailenco/msgpack@v4.0.4+incompatible: can only use path@version syntax with 'go get'
can't load package: package foo/build/pkg/mod/github.com/vmihailenco/msgpack@v4.0.4+incompatible/codes: can only use path@version syntax with 'go get'
can't load package: package foo/build/pkg/mod/go.etcd.io/bbolt@v1.3.2: can only use path@version syntax with 'go get'
can't load package: package foo/build/pkg/mod/go.etcd.io/bbolt@v1.3.2/cmd/bbolt: can only use path@version syntax with 'go get'
can't load package: package foo/build/pkg/mod/gopkg.in/check.v1@v1.0.0-20180628173108-788fd7840127: can only use path@version syntax with 'go get'
[List of packages removed]
matthijs@grubby:~/foo$ cat go.mod
module foo
require (
github.com/Sereal/Sereal v0.0.0-20190409170602-963d7e218945 // indirect
github.com/asdine/storm v2.1.2+incompatible
github.com/golang/protobuf v1.3.1 // indirect
github.com/golang/snappy v0.0.1 // indirect
github.com/kr/pretty v0.1.0 // indirect
github.com/stretchr/testify v1.3.0 // indirect
github.com/vmihailenco/msgpack v4.0.4+incompatible // indirect
go.etcd.io/bbolt v1.3.2 // indirect
golang.org/x/sys v0.0.0-20190410235845-0ad05ae3009d // indirect
gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127 // indirect
)
I'm also not sure what theys path@version error messages mean (I did not get them in my actual project, I think). For now, I'm ignoring them.
An additional source of confusion is how the dependencies within the storm package are handled. In particular, why is github.com/asdine/storm/codec/sereal included? AFAIK, running go mod why should tell me (but again downloads more dependencies?).
matthijs@grubby:~/foo$ go mod why github.com/asdine/storm/codec/sereal
go: finding github.com/DataDog/zstd v1.3.8
go: finding google.golang.org/appengine/datastore latest
go: downloading github.com/DataDog/zstd v1.3.8
go: finding google.golang.org/appengine v1.5.0
go: downloading google.golang.org/appengine v1.5.0
go: finding github.com/golang/protobuf v1.2.0
go: finding golang.org/x/net v0.0.0-20180724234803-3673e40ba225
go: finding golang.org/x/text v0.3.0
go: downloading golang.org/x/net v0.0.0-20180724234803-3673e40ba225
# github.com/asdine/storm/codec/sereal
foo
github.com/asdine/storm
github.com/asdine/storm/codec
github.com/asdine/storm/codec.test
github.com/asdine/storm/codec/sereal
matthijs@grubby:~/foo$ grep -r sereal build/pkg/mod/github.com/asdine/storm@v2.1.2+incompatible/
build/pkg/mod/github.com/asdine/storm@v2.1.2+incompatible/README.md:You can easily implement your own `MarshalUnmarshaler`, but Storm comes with built-in support for [JSON](https://godoc.org/github.com/asdine/storm/codec/json) (default), [GOB](https://godoc.org/github.com/asdine/storm/codec/gob), [Sereal](https://godoc.org/github.com/asdine/storm/codec/sereal), [Protocol Buffers](https://godoc.org/github.com/asdine/storm/codec/protobuf) and [MessagePack](https://godoc.org/github.com/asdine/storm/codec/msgpack).
build/pkg/mod/github.com/asdine/storm@v2.1.2+incompatible/README.md: "github.com/asdine/storm/codec/sereal"
build/pkg/mod/github.com/asdine/storm@v2.1.2+incompatible/README.md:var serealDb, _ = storm.Open("sereal.db", storm.Codec(sereal.Codec))
build/pkg/mod/github.com/asdine/storm@v2.1.2+incompatible/codec/example_test.go: "github.com/asdine/storm/codec/sereal"
build/pkg/mod/github.com/asdine/storm@v2.1.2+incompatible/codec/example_test.go: var serealDb, _ = storm.Open("sereal.db", storm.Codec(sereal.Codec))
build/pkg/mod/github.com/asdine/storm@v2.1.2+incompatible/codec/example_test.go: fmt.Printf("%T\n", serealDb.Codec())
build/pkg/mod/github.com/asdine/storm@v2.1.2+incompatible/codec/example_test.go: // *sereal.serealCodec
build/pkg/mod/github.com/asdine/storm@v2.1.2+incompatible/codec/sereal/sereal.go:// Package sereal contains a codec to encode and decode entities using Sereal
build/pkg/mod/github.com/asdine/storm@v2.1.2+incompatible/codec/sereal/sereal.go:package sereal
build/pkg/mod/github.com/asdine/storm@v2.1.2+incompatible/codec/sereal/sereal.go: "github.com/Sereal/Sereal/Go/sereal"
build/pkg/mod/github.com/asdine/storm@v2.1.2+incompatible/codec/sereal/sereal.go:const name = "sereal"
build/pkg/mod/github.com/asdine/storm@v2.1.2+incompatible/codec/sereal/sereal.go:var Codec = new(serealCodec)
build/pkg/mod/github.com/asdine/storm@v2.1.2+incompatible/codec/sereal/sereal.go:type serealCodec int
build/pkg/mod/github.com/asdine/storm@v2.1.2+incompatible/codec/sereal/sereal.go:func (c serealCodec) Marshal(v interface{}) ([]byte, error) {
build/pkg/mod/github.com/asdine/storm@v2.1.2+incompatible/codec/sereal/sereal.go: return sereal.Marshal(v)
build/pkg/mod/github.com/asdine/storm@v2.1.2+incompatible/codec/sereal/sereal.go:func (c serealCodec) Unmarshal(b []byte, v interface{}) error {
build/pkg/mod/github.com/asdine/storm@v2.1.2+incompatible/codec/sereal/sereal.go: return sereal.Unmarshal(b, v)
build/pkg/mod/github.com/asdine/storm@v2.1.2+incompatible/codec/sereal/sereal.go:func (c serealCodec) Name() string {
build/pkg/mod/github.com/asdine/storm@v2.1.2+incompatible/codec/sereal/sereal_test.go:package sereal
matthijs@grubby:~/foo$ cat go.mod
module foo
require (
github.com/DataDog/zstd v1.3.8 // indirect
github.com/Sereal/Sereal v0.0.0-20190409170602-963d7e218945 // indirect
github.com/asdine/storm v2.1.2+incompatible
github.com/golang/protobuf v1.3.1 // indirect
github.com/golang/snappy v0.0.1 // indirect
github.com/kr/pretty v0.1.0 // indirect
github.com/stretchr/testify v1.3.0 // indirect
github.com/vmihailenco/msgpack v4.0.4+incompatible // indirect
go.etcd.io/bbolt v1.3.2 // indirect
golang.org/x/sys v0.0.0-20190410235845-0ad05ae3009d // indirect
google.golang.org/appengine v1.5.0 // indirect
gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127 // indirect
)
This shows codec/sereal is pulled in by codec.test (aka codec_test defined in example_test.go). A grep of the storm source confims that it is only included from the test code.
However, the codec.test package is not actually present in go list output:
matthijs@grubby:~/foo$ go list all 2> /dev/null |grep asdine
github.com/asdine/storm
github.com/asdine/storm/codec
github.com/asdine/storm/codec/gob
github.com/asdine/storm/codec/internal
github.com/asdine/storm/codec/json
github.com/asdine/storm/codec/msgpack
github.com/asdine/storm/codec/protobuf
github.com/asdine/storm/codec/sereal
github.com/asdine/storm/index
github.com/asdine/storm/internal
github.com/asdine/storm/q
If codec.test is not present, why is its sereal dependency included anyway? Or is there a difference between package names and (import) paths here?