How to `go get` private repos using SSH key auth instead of password auth.

goGetPrivate.md

Set GOPRIVATE to match your github organization

Cloning the repo using one of the below techniques should correctly but you may still getting an unrecognized import error.

As it stands for Go v1.13, I found in the doc that we should use the GOPRIVATE variable like so:

GOPRIVATE=github.com/ORGANISATION_OR_USER_NAME go get -u -f github.com/ORGANISATION_OR_USER_NAME/REPO_NAME

The 'go env -w' command (see 'go help env') can be used to set these variables for future go command invocations.

How to go get private repos using SSH key auth instead of password auth.

$ git config --global url."git@github.com:".insteadOf "https://github.com/"
$ git config --global url."git@gitlab.com:".insteadOf "https://gitlab.com/"
$ cat ~/.gitconfig
[url "git@github.com:"]
	insteadOf = https://github.com/
[url "git@gitlab.com:"]
	insteadOf = https://gitlab.com/
$ go get -u -f github.com/private/repo && echo Success!
Success!

In Git ssh://git@host/resource, git+ssh://git@host/resource, ssh+git://git@host/resource and git@host:resource are all the same protocol.

💡 NOTE: go get uses HTTPS when cloning a repository. So configuring insteadOf rewriting rules will cause problems with go get -u github.com/private/repo later on, since a check at update time to verify that the local package's remote repository matches that of its custom import. This change was introduced in Go 1.4.

The workaround for that rewriting check would be to use go get -u -f github.com/private/repo:

From the go get options here:

The -f flag, valid only when -u is set, forces get -u not to verify that each package has been checked out from the source control repository implied by its import path. This can be useful if the source is a local fork of the original.

Sources:

• https://help.github.com/articles/which-remote-url-should-i-use #You can also use SSH agent forwarding with your deploy script to avoid managing keys on the server.
• http://golang.org/doc/faq #Why does "go get" use HTTPS when cloning a repository?
• https://www.kernel.org/pub/software/scm/git/docs/git-config.html #url.<base>.insteadOf

How to go get private repos using HTTPS via Personal Access Tokens

An alternative to using git@github.com is to generate a personal access token on your GitHub account (or for a service account), grant it repo access, and then use the following instead:

# Github
git config --global url."https://${GITHUB_TOKEN}:x-oauth-basic@github.com/".insteadOf "https://github.com/"

# Gitlab personal access token
git config --global url."https://gitlab-ci-token:${GITLAB_PERSONAL_ACCESS_TOKEN}@gitlab.com/".insteadOf "https://gitlab.com/"

# BitBucket is a weird place. You need user slug rather than username
git config --global url."https://user%40bitbucket.com:${BITBUCKET_ACCESS_TOKEN}@bitbucket.com/".insteadOf "https://bitbucket.com/"

You could also just store credentials in a credential helper like the ~/.git-credentials file.

Both should still work with go get -u, and also works with Docker builds.

Git Credential Manager (GCM) alternative

Git Credential Manager (GCM) is another way to store your credentials securely and connect to GitHub over HTTPS. With GCM, you don't have to manually create and store a personal access token, as GCM manages authentication on your behalf, including 2FA (two-factor authentication).

1. Install Git using Homebrew:

brew install git

2. Install GCM using Homebrew:

brew install --cask git-credential-manager

For macOS, you don't need to run git config because GCM automatically configures Git for you.

The next time you clone an HTTPS URL that requires authentication, Git will prompt you to log in using a browser window. You may first be asked to authorize an OAuth app. If your account or organization requires two-factor auth, you'll also need to complete the 2FA challenge.

Once you've authenticated successfully, your credentials are stored in the macOS keychain and will be used every time you clone an HTTPS URL. Git will not require you to type your credentials in the command line again unless you change your credentials.

For more information or to report issues with GCM, see the official GCM docs at "Git Credential Manager."

Nice

And helpful

it works!

current known configuration for MFA is here config available here: golang/go#49515 (comment)

Thanks for writing this up. Question for you. - let's say I have a project that uses a module that's in a private repo. I set my GOPRIVATE as you describe.

At the git config step (git config --global url."git@github.com:".insteadOf "https://github.com/"), can't I replace the --global flag with --local and make that rewrite specific only to my project?

@shuchow It depends on your use case. Inside the context of that configured git repository, it will work fine. However, if you are ever trying to use the Go tools to reference the private module outside of the context of that git configured repository, it will fail. For instance, when you might go install <modulename> a binary from that private repository, or if you have another git repository that is not configured.

For me, I always want ssh, so git config --global has less potential to be problematic.

for me I was incorrectly using a delimiter of character : instead of the comma when defining env var

export GOPRIVATE=github.com/foo/bar_private_repoAAA,github.com/foo/bar_private_repoBBB

After change url to git@gitlab.com I can clone the repo but cannot go get
Got this error:

	remote: 
	remote: ========================================================================
	remote: 
	remote: ERROR: The project you were looking for could not be found or you don't have permission to view it.
	
	remote: 
	remote: ========================================================================
	remote: 
	fatal: Could not read from remote repository.
	
	Please make sure you have the correct access rights
	and the repository exists.

Does anyone have the same error like that?

@hmit208 did you do this first:

# set GITLAB_PERSONAL_ACCESS_TOKEN environment variable to Gitlab personal access token
git config --global url."https://gitlab-ci-token:${GITLAB_PERSONAL_ACCESS_TOKEN}@gitlab.com/".insteadOf "https://gitlab.com/"

After change url to git@gitlab.com I can clone the repo but cannot go get Got this error:

	remote: 
	remote: ========================================================================
	remote: 
	remote: ERROR: The project you were looking for could not be found or you don't have permission to view it.
	
	remote: 
	remote: ========================================================================
	remote: 
	fatal: Could not read from remote repository.
	
	Please make sure you have the correct access rights
	and the repository exists.

Does anyone have the same error like that?

You are using sub-groups in Gitlab? If that's the case you should follow this issue:
https://gitlab.com/gitlab-org/gitlab/-/issues/36354

Extra thing to try (that worked for me) - ssh-add ~/.ssh/github

This is great, thanks a lot for publishing this!

Also a good related article about getting private Go modules to work with Github actions:
https://aran.dev/posts/github-actions-go-private-modules/

I have found that go get private-repo.com/path/myModule still tries to use HTTPS but go get private-repo.com/path/myModule.git respects the insteadOf configuration correctly (note the ".git" at the end). Lots of older writeups say it worked without the ".git" so I'm wondering if it was a change within the last few Go versions...