Skip to content

Instantly share code, notes, and snippets.

@leomelzer
Last active March 9, 2022 19:18
Show Gist options
  • Star 12 You must be signed in to star a gist
  • Fork 3 You must be signed in to fork a gist
  • Save leomelzer/20558eca5ed5ae66837a044e0339a45f to your computer and use it in GitHub Desktop.
Save leomelzer/20558eca5ed5ae66837a044e0339a45f to your computer and use it in GitHub Desktop.
Install private NPM dependencies using git+ssh on Now. Based on https://zeit.co/blog/build-env to work around missing support for git+ssh dependencies in now@2 (https://github.com/zeit/now-builders/issues/49)
# This is a multi-stage build to not spill the contents of the deploy_key
FROM mhart/alpine-node:10 as base
# We need git and openssh to resolve `git+ssh` links in package.json
RUN apk update \
&& apk add git openssh
WORKDIR /usr/src
COPY package*.json ./
# Pass through Now build-env as Docker build-arg
ARG EXAMPLE_DEPLOY_KEY
# Setup key file during build stage
RUN mkdir -p ~/.ssh
RUN echo $EXAMPLE_DEPLOY_KEY | base64 -d > ~/.ssh/deploy_key
RUN chmod 600 ~/.ssh/deploy_key
# Add github.com to known hosts to avoid error
RUN ssh-keyscan -t rsa github.com >> ~/.ssh/known_hosts
# Install dependencies (incl. `git+ssh` links)
ENV GIT_SSH_COMMAND "ssh -i ~/.ssh/deploy_key"
RUN npm install
# Copy resulting output and build app
COPY . .
RUN npm run now-build
# Finally, assemble the running image
FROM mhart/alpine-node:base-10
WORKDIR /usr/src
# Now v1 expects static assets at `/public`
RUN mkdir /public
COPY --from=base /usr/src/build /public
{
"version": 1,
"public": false,
"name": "example-project",
"alias": "example-project.peerigon.io",
"type": "static",
"scale": {
"bru1": {
"min": 0,
"max": 1
}
},
"static": {
"rewrites": [
{
"source": "**",
"destination": "/index.html"
}
]
},
"env": {
"NODE_ENV": "production"
},
"build": {
"env": {
"EXAMPLE_DEPLOY_KEY": "@example-deploy-key"
}
}
}
{
"name": "example-project",
"version": "0.1.0",
"private": true,
"dependencies": {
"private-module": "git+ssh://git@github.com/peerigon/private-module.git",
"prop-types": "^15.6.2",
"react": "^16.6.0",
"react-dom": "^16.6.0",
"react-scripts": "2.0.5"
},
"scripts": {
"start": "react-scripts start",
"build": "react-scripts build",
"test": "react-scripts test",
"now-build": "npm run build"
}
}

Setup

  1. Generate a new deploy key. You should not add a password (leave blank) since you would have to provide it every time, rendering the automatization useless. Save the private part of the key to the file my-key.key.
  2. Add the generated key as deploy key to your GitHub repository. You should make sure to keep the key read-only (default).
  3. Add a secret to Now which can be used in the Deployment:
$ now secret add example-deploy-key "$(cat my-key.key | base64)"

Encoding the file using base64 is no "Security through obscurity" but instead ensures there are no encoding problems (e.g. newlines \n cause errors in the Now CLI). We trust Now to transmit and store the secret values in a secure manner.

  1. Reference the secret as build-time environment variable (--build-arg in Docker).

Part of now.json:

{
  "env": {
    "NODE_ENV": "production"
  },
  "build": {
    "env": {
      "EXAMPLE_DEPLOY_KEY": "@example-deploy-key"
    }
  }
}

Putting the @ in front of an identifier tries to resolve the secret with the same name.

We can now access the value in the Dockerfile:

ARG EXAMPLE_DEPLOY_KEY
RUN echo $EXAMPLE_DEPLOY_KEY

NOTE: You can only read secrets from the current scope, e.g. if you created the secret with a team scope (now --team peerigon secret add some-secret-name some-secret-value you'll need to run the now command with the same scope.


I hope this helps someone!

@pawelotto
Copy link

pawelotto commented Jul 16, 2019

@leomelzer Thanks for replying!

I've been struggling for a while with this trying to install from GitLab.

I noticed NOW has ssh and git installed by default on their machines so I tried to add necessary deploy SSH keys for GitLab in the preinstall script:

ssh-keyscan -t rsa gitlab.com >> ~/.ssh/known_hosts && eval `ssh-agent` && ssh-add .ssh/my_gitlab_repo_deploy_key

Everything ran smoothly in deploy until I got the Host key verification error:

yarn install v1.16.0
$ mkdir -p ~/.ssh && ssh-keyscan gitlab.com >> ~/.ssh/known_hosts && eval `ssh-agent` && ssh-add .ssh/my_gitlab_repo_deploy_key
# gitlab.com SSH-2.0-OpenSSH_7.2p2 Ubuntu-4ubuntu2.8
Agent pid 94
Identity added: .ssh/my_gitlab_repo_deploy_key (.ssh/my_gitlab_repo_deploy_key)
[1/4] Resolving packages...
[2/4] Fetching packages...
error Command failed.
Exit code: 128
Command: git
Arguments: ls-remote --tags --heads git@gitlab.com:myrepo.git
Directory: /tmp/666c4c3e
Output:
Host key verification failed.

fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

I also tried removing the host with ssh-keygen -R but to no avail. I verified that ~/.ssh/known_hosts is indeed being correctly created.
It seems like ssh on NOW machine is reading a different known_hosts file from that I created in preinstall or they even run a separate machine during deployment that is using different config, I wasn't able to figure that out.

Do you have any idea what could go wrong in this setup?

Thanks for the private npm token tip I will try this.

@leomelzer
Copy link
Author

@pawelotto Sure thing! These kind of problems can be very annoying but can be solved :) Hm, but your approach sounds very well 👌

I found it handy to set GIT_SSH_COMMAND="ssh -v" as environment variable which basically modifies the ssh command git will use. This would e.g. make the output super verbose. That helped me in some case where it would actually not try my deploy key file where I expected it to. Passing ssh -vvv increases the verbosity, I found one v to be enough for my cases. You could pass other arguments, too! Hope that helps debugging the problem.

@pawelotto
Copy link

@leomelzer Thanks bro! I'm the meantime I solved this by using deploy tokens and everything works fine now. Cheers!

@leomelzer
Copy link
Author

@pawelotto Great it's working! You mean those? https://docs.gitlab.com/ee/user/project/deploy_tokens/

Maybe you can document your Gitlab solution in your own gist.github.com for people coming from Google or so? :)

@pawelotto
Copy link

@leomelzer Yes, exactly that. I'm setting deploy tokens to be read_only and separately for each private repo I'm referencing in package.json

It was a cumbersome trial & error process to figure it out and to make it work with NOW, you're right I could share my solution with the community. Will attach a gist when I'm at my desktop.

Cheers mate

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment