Skip to content

Instantly share code, notes, and snippets.

@azu
Last active Nov 3, 2021
Embed
What would you like to do?
GitHub Package Registry and Npm Registry for same scoped does not work @ 2020-01-10

📝 I submit the same text to GitHub Support.


I want to use same scope for npm package and GitHub Package Registry.

For example, @org scope exist in npm and GitHub.

  • @org/foo-public package is public on npm registry
  • @org/bar-private package is private on GitHub Package Registry

This situation is often the case. @org/foo-public is OSS library and @org/bar-private is internal library.

But, I can not use both at same time.

When I used following npmrc config, I could not refer to @org/foo-public. Because https://npm.pkg.github.com/foo does not proxy to npm I can only refer to @org/bar-private

//npm.pkg.github.com/:_authToken=YYY
registry="https://npm.pkg.github.com/org"

Reproduce Example

.npmrc

//registry.npmjs.org/:_authToken=XXX # login my npm account
//npm.pkg.github.com/:_authToken=YYY # login my github account

registry=https://npm.pkg.github.com/azu

Refs: https://help.github.com/en/github/managing-packages-with-github-packages/configuring-npm-for-use-with-github-packages#installing-packages-from-other-organizations

Example Pacakge

Actual

I can not fetch @azu/testtest package from npm reigstry. Because, GitHub Package Registry does not proxy to npm registry about scoped package.

$  npm info @azu/testtest
npm ERR! code E404
npm ERR! 404 '@azu/testtest' is not in the npm registry.
npm ERR! 404 You should bug the author to publish it
npm ERR! 404 (or use the name yourself!)
npm ERR! 404
npm ERR! 404 Note that you can also install from a
npm ERR! 404 tarball, folder, http url, or git url.
npm ERR! 404
npm ERR! 404  '@azu/testtest@latest' is not in the npm registry.
npm ERR! 404 You should bug the author to publish it (or use the name yourself!)
npm ERR! 404
npm ERR! 404 Note that you can also install from a
npm ERR! 404 tarball, folder, http url, or git url.

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/azu/.npm/_logs/2020-01-10T13_54_07_224Z-debug.log

@azu scope work only for GitHub Package Registry

$ npm info @azu/npm-github-package-example

@azu/npm-github-package-example@2.3.0 | MIT | deps: none | versions: 9
npm registry to GitHub Package Registry example.
https://github.com/azu/npm-github-package-example

dist
.tarball: https://npm.pkg.github.com/download/@azu/npm-github-package-example/2.3.0/3bdb8322c3344012bcf5ab5819f433fc1e2029ca5a9cf42e8580a2dd0086d9e1
.shasum:

dist-tags:
latest: 2.3.0

published over a year ago by undefined <undefined>

Expected

Both is work.

$ npm info @azu/testtest

@azu/testtest@1.0.0 | MIT | deps: none | versions: 2

dist
.tarball: https://registry.npmjs.org/@azu/testtest/-/testtest-1.0.0.tgz
.shasum: a8ed44b4e3ea61011fc257a238aa36306ef316f3
.integrity: sha512-UPyAXnx64pWO14oUGwJlTm1f1VAIMOWMKl5YlXCXETxfJSBxjHi0WR2sKk2JVls1CacLttmv0FwXXwFufNJvIg==
.unpackedSize: 230 B

maintainers:
- azu <azuciao@gmail.com>

dist-tags:
a1: 1.0.1      a: 1.0.0       latest: 1.0.0

published 6 months ago by azu <azuciao@gmail.com>

$ npm info @azu/npm-github-package-example

@azu/npm-github-package-example@2.3.0 | MIT | deps: none | versions: 9
npm registry to GitHub Package Registry example.
https://github.com/azu/npm-github-package-example

dist
.tarball: https://npm.pkg.github.com/download/@azu/npm-github-package-example/2.3.0/3bdb8322c3344012bcf5ab5819f433fc1e2029ca5a9cf42e8580a2dd0086d9e1
.shasum:

dist-tags:
latest: 2.3.0

published over a year ago by undefined <undefined>

Question

Is this bug of GitHub Package Registry?

I think that registry="https://npm.pkg.github.com/org should proxy to npm's @org/* package. This feature is called Uplinks in verdaccio.

It seem that yarnpkg registry support similar proxy feature.

GitHub Package Registry does not work.

$ npm info @azu/testtest --registry "https://npm.pkg.github.com/azu"
npm ERR! code E404

yarnpkg registry does work.

$ npm info @azu/testtest --registry "https://registry.yarnpkg.com"

@azu/testtest@1.0.0 | MIT | deps: none | versions: 2

dist
.tarball: https://registry.npmjs.org/@azu/testtest/-/testtest-1.0.0.tgz
.shasum: a8ed44b4e3ea61011fc257a238aa36306ef316f3
.integrity: sha512-UPyAXnx64pWO14oUGwJlTm1f1VAIMOWMKl5YlXCXETxfJSBxjHi0WR2sKk2JVls1CacLttmv0FwXXwFufNJvIg==
.unpackedSize: 230 B

maintainers:
- azu <azuciao@gmail.com>

dist-tags:
a1: 1.0.1      a: 1.0.0       latest: 1.0.0

published 6 months ago by azu <azuciao@gmail.com>

Summary

I've tested variable patterns for combination registries.

Pattern A

registry="https://npm.pkg.github.com/ORG"

Pattern B

registry="https://registry.npmjs.org"
@ORG:registry="https://npm.pkg.github.com/ORG"

Pattern C

registry="https://registry.npmjs.org"
@ORG:registry="https://npm.pkg.github.com"

Related

Pattern D

registry=https://npm.pkg.github.com/ORG
@ORG:registry=https://registry.npmjs.org

Results

Pattern non-scoped package Npm @ORG/* GitHub @ORG/*
A ok ng ok
B ok ng ok
C ok ng ok
D ok ok ng
@azu
Copy link
Author

azu commented Mar 26, 2020

@pinalbhatt No response from GitHub.
I've moved all private packages(npm private registry) to GitHub Registry.
Also, I've published public pacpages(npm public registry) to GitHub Registry too.

@Alexandre-cibot
Copy link

Alexandre-cibot commented Apr 20, 2020

I got the exact same issue.
Decided to move my private package from npm to the Github Registery either.

@aendra-rininsland
Copy link

aendra-rininsland commented Apr 24, 2020

I'm in exactly the same boat and ended up doing what @azu did — copy everything into both package registries — which is such a submoronic solution, but the only one that seems viable at the moment.

I'm so, so angry at how badly this all works...

@mholtzhausen
Copy link

mholtzhausen commented May 8, 2020

I think their intention is to move all private repos onto github registry, deprecate npm enterprise and use npm only for public repos. They will have to solve this scoping issue before they can do that.

@Oloompa
Copy link

Oloompa commented Jun 12, 2020

But it will not solve the issue if you have same org name for different git provider.
In my case i use both gitlab and github. Same org name. Same issue...

@thani-sh
Copy link

thani-sh commented Jul 6, 2020

I had the same issue. I tried publishing to both registries first but ended up using a different scope for private packages.

@azu
Copy link
Author

azu commented Jul 9, 2020

📝 I've submit this issue to GitHub support again.
(Recently GitHub spport got new ticket system https://support.github.com/tickets/ . I want to track this issue in new ticket system)

GitHub support told me a workaround using git dependencies.

Example: https://github.com/jcansdale-test/consume-private-npm-package

Reply from GitHub support:

A possible alternative is to use a repository rather than a package reference (npm has built in support for referencing GitHub repositories). For example, instead of using @org/bar-private@version you can use org/bar-private#version. This will reference a private repository rather than a private package.

Try making your .npmrc like this:

//registry.npmjs.org/:_authToken=XXX # login my npm account

You should find you can do both of these:

$ npm i @azu/testtest # install using a package reference

+ @azu/testtest@1.0.0
added 1 package from 1 contributor and audited 100 packages in 3.201s
found 0 vulnerabilities

$ npm i azu/npm-github-package-example # install using a repository reference

+ @azu/npm-github-package-example@2.3.2
updated 1 package and audited 100 packages in 2.85s
found 0 vulnerabilities

If you create a release tag for each version, you'll find you can also do this:

$ npm i azu/npm-github-package-example#2.0.0

+ @azu/npm-github-package-example@2.0.1
updated 1 package and audited 100 packages in 2.583s
found 0 vulnerabilities

If you need to consume these private packages from a GitHub Actions workflow, you can create and install a read-only deploy key.

Here is an example repository that consumes a private package:
https://github.com/jcansdale-test/consume-private-npm-package

Could you let me know what you think of using this approach?

I think that I can not use this workaround in some case.
I replied as following:

Reply from @azu:

Yes, This workaround can avoid some issue, but a lof of repository does not includes built files.

For example, if you use TypeScript, the repository only commit *.ts file that is source code in the git repository.
The *.js files are generated by TypeScript compiler when prepublish phase.
As a result, The repository does not include *.js files that are built by TypeScript.
In this case, I can not to use GitHub repositories as dependencies.

    https://docs.npmjs.com/misc/scripts#prepublish-and-prepare
    https://docs.npmjs.com/misc/scripts#use-cases

This is common use case of Package Registry.

@mahnunchik
Copy link

mahnunchik commented Dec 4, 2020

I've faced with the same issue 😢

@mahnunchik
Copy link

mahnunchik commented Dec 4, 2020

Related RFCs npm/rfcs#217 and npm/rfcs#211

@pinalbhatt
Copy link

pinalbhatt commented Mar 26, 2021

hi friends, do we have any solution or workaround of this problem?

@katlimruiz
Copy link

katlimruiz commented Jun 7, 2021

The solution should be that to configure an upstream link between Github Packages and npm, this way the request first goes to GP, if not found, it goes to npm, and the cycle is completed.

That is what Azure Devops has, aaaaanndddd I thought GP was in a better shape :|

By the way I also tried this

@myreg:registry=https://npm.pkg.github.com
@myreg:registry=https://registry.npmjs.org
always-auth=true

and it did not work

@vis97c
Copy link

vis97c commented Jul 3, 2021

According to this you have to use another format for the proxy to work.

registry=https://npm.pkg.github.com/YOUR_ORG

But i doesn't work either. I was considering to move my public packages to github but i'm using unpkg to serve some static files, so i would lost that if i make the switch. Has anyone found a solution for this?

@jhnns
Copy link

jhnns commented Sep 3, 2021

Not the perfect solution, but maybe enough for your use case:

You can override the registry when installing the package, e.g.:

npm install --registry https://registry.npmjs.org @your-org/public-package

Once the package is installed, the correct registry is stored in the package-lock.json. You should then be able to just run npm install (e.g. during CI) and it will pick the respective registry.

@jhnns
Copy link

jhnns commented Sep 3, 2021

In case you're using semantic-release, you might also take a look at: https://github.com/amanda-mitchell/semantic-release-npm-multiple

@shawnmclean
Copy link

shawnmclean commented Nov 3, 2021

Damn, I can't go the lock file route. My code is semi-processed (pulling a package out of a monorepo and building it individually).

Any other solution?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment