Deploy Google Cloud Functions: GitLab CI/CD Pipeline Config File

.gitlab-ci.yml

# Update Jan 2024
# Deploying Cloud Functions is much simpler than it was 6 years ago.
# I'm leaving the gist in it's original 2018 state for now,
# but skip the the recent comments below for a simpler solution.

variables:
  GCP_ZONE: us-central1-a

stages:
  - npm-install
  - push

npm-install:
  image: node:8-alpine
  stage: npm-install
  only:
    - master
  script:
    - cd replaceWithYourCloudFunction
    - npm install
  artifacts:
    paths:
      - replaceWithYourCloudFunction/node_modules/

push:
  stage: push
  image: docker:latest
  only:
    - master
  dependencies:
    - npm-install
  script:
    # Install CA certs, openssl to https downloads, python for gcloud sdk
    - apk add --update make ca-certificates openssl python
    - update-ca-certificates

    # Download and install Google Cloud SDK
    - wget https://dl.google.com/dl/cloudsdk/release/google-cloud-sdk.tar.gz
    - tar zxvf google-cloud-sdk.tar.gz && ./google-cloud-sdk/install.sh --quiet --usage-reporting=false --path-update=true
    - PATH="google-cloud-sdk/bin:${PATH}"
    - gcloud --quiet components update
    - gcloud components install beta

    # Authenticate using the service account stored here: https://gitlab.com/groups/{YOUR_GITLAB_PROJECT}/-/settings/ci_cd
    - echo $GCLOUD_SERVICE_KEY > ${HOME}/gcloud-service-key.json
    - gcloud auth activate-service-account --key-file ${HOME}/gcloud-service-key.json
  
    # Deploy
    - gcloud beta functions deploy replaceWithYourCloudFunction --source=./replaceWithYourCloudFunction --trigger-http

That helped me a lot. Tks.

Would you mind to explain why you need docker:dind as service and why you used docker as image for deploy the Function? Couldn't you just use a Debian image for that and remove the service?

Hi @troyharvey
Thanks a lot for this gist. It's exactly what I was looking for.
Unfortunately I don't have a lot of experience with GitLab CI/CD and therefore couldn't manage to get this working.

Since my project only consists of a simple main.py file along with a requirements.txt and some additional files (README.md, .gitignore, ...) I suppose I won't need all the npm-install stuff in your receipt? I deleted it, pushed changes to my repository and got the Job running. It failed although :-(

My jobs always end with a error message like this:

$ echo $GCLOUD_SERVICE_KEY > ${HOME}/gcloud-service-key.json
$ gcloud auth activate-service-account --key-file ${HOME}/gcloud-service-key.json
Activated service account credentials for: [gibz-one50@appspot.gserviceaccount.com]
$ gcloud config set project $GCP_PROJECT_ID
Updated property [core/project].
$ gcloud functions deploy send_2fa_sms --trigger-http --runtime=python37 --region=${GCP_ZONE}
ERROR: (gcloud.functions.deploy) Failed to upload the function source code to signed url: https://storage.googleapis.com/gcf-upload-europe-west1-f84788ab-087c-4c71-b28c-1d04b7e3fd74/202eed5b-3a52-4dd6-bff4-fed4dbecc8e7.zip?GoogleAccessId=service-576747503219@gcf-admin-robot.iam.gserviceaccount.com&Expires=1541631358&Signature=pWvPNbexuu7wpvJo%2F9c4Ppqp%2BRU%2B4L3JbGmtWljzT7has%2BuTIrTsLYuCmmQMh3fucL3BcW%2FQzLuOPrBjSWRbBk5I7gXMk9Jj89icBkLOqg%2Fh2WrjyY0Uger0PXv9BJwOKsp4EIlOE%2BFPPcc5q0O8CmXFhguyuNDCYxS%2Bk7a14BcVWkpdS3P9Y0EzD%2FTfNpmqS4iXFvYVHfx%2F%2BowB9H4TCMzfDRUeBI53mMbmHNkvcJmhcBg03nnmn1BOku1ibgXI7sl%2Bhiv%2B9qL1nJNWaRmAHhCBTC7%2BJowTsDdNtnO6Q0BVF%2Fv4tOODEewJnSp06R9ihgMh9BJrJRQs4vxYpVeovQ%3D%3D. Status: [400:<?xml version='1.0' encoding='UTF-8'?><Error><Code>EntityTooLarge</Code><Message>Your proposed upload is larger than the maximum object size specified in your Policy Document.</Message><Details>Content-length exceeds upper bound on range</Details></Error>]
ERROR: Job failed: exit code 1

Do you have any hints for me what I could do to get it running? Where do you suppose lies the problem?

Kind regards,
Peter

Thanks.

You don't need Dind for this. Also, you don't have to set project (when you authenticate using Service Account it automatically knows the project).

bingo - there's a container with SDK from Google - https://hub.docker.com/r/google/cloud-sdk

You need to add a --quiet also here:
- tar zxvf google-cloud-sdk.tar.gz && ./google-cloud-sdk/install.sh --usage-reporting=false --path-update=true
(at least I needed to do so)

Thanks @JannikZed and @viggy28. Does it look consistent with what you are using in production now?

1. Removed the dind service
2. Added --quiet on the install.sh step
3. Removed the gcloud config set project $GCP_PROJECT_ID

looks like our setup now, thank you :) we use gsutil afterwards to load our machine-learning models from a storage bucket. Thank you for this gist. Reduced my effort a lot!!

hi @troyharvey,

Do you know why I got this error

constants.js
constants.js.map
db
dialogFlowApp.js
dialogFlowApp.js.map
enums.js
enums.js.map
index.js
index.js.map
intents
models
templates
utils

$ gcloud beta functions deploy fulfillment --source=./voice/google-assistant/webhook/functions --trigger-http --runtime nodejs8
Created .gcloudignore file. See `gcloud topic gcloudignore` for details.
Deploying function (may take a while - up to 2 minutes)...
..............failed.
ERROR: (gcloud.beta.functions.deploy) OperationError: code=3, message=Function failed on loading user code. Error message: File lib/index.js that is expected to define function doesn't exist.

inside source folder, I have index.js file but when gcloud deploy I got above error message

This should be enough for deployment:

deploy to gcloud:
  image: google/cloud-sdk:latest
  script:
    - echo $GCLOUD_SERVICE_KEY > /tmp/$CI_PIPELINE_ID.json
    - gcloud auth activate-service-account --key-file /tmp/$CI_PIPELINE_ID.json
    - gcloud functions deploy function-name --runtime nodejs8 --entry-point handler --trigger-http
    - rm /tmp/$CI_PIPELINE_ID.json

@muratcorlu
If you set the service key as a file variable instead of a normal variable, then you don't need to create the tmp file, removing first and last step.

@jligeza Good information, thanks. So it'll be that much simple:

deploy to gcloud:
  image: google/cloud-sdk:latest
  script:
    - gcloud auth activate-service-account --key-file $GCLOUD_SERVICE_KEY_FILE
    - gcloud functions deploy function-name --runtime nodejs8 --entry-point handler --trigger-http

Hello @muratcorlu,

First thanks for sharing the CI/CD :)
I'll share my tests at this moment to help if other needs them.

For me, the project ID is still mandatory, else I get a:

ERROR: (gcloud.app.deploy) The required property [project] is not currently set.
You may set it for your current workspace by running:
  $ gcloud config set project VALUE
or it can be set temporarily by the environment variable [CLOUDSDK_CORE_PROJECT]
ERROR: Job failed: exit code 1

(Which is still strange, cause It's clearly indicated in the JSON as project_id).

Also not sure of what @jiligeza said because i'm not sure of what a "file variable" is (maybe a setting in giltab CI I did not found) and passing it without create a file still cause an error as:

 ERROR: (gcloud.auth.activate-service-account) argument --key-file: expected one argument
Usage: gcloud auth activate-service-account [ACCOUNT] --key-file=KEY_FILE [optional flags]

For the moment i'm fighting to find why the CI/CD successfully deploy on staging, but not on prod. Same key, same code and I get a:

ERROR: (gcloud.auth.activate-service-account) Could not read json file /tmp/124294711.json: Expecting value: line 2 column 1 (char 1)
ERROR: Job failed: exit code 1

While on dev branch

$ gcloud auth activate-service-account --key-file /tmp/$CI_PIPELINE_ID.json
Activated service account credentials for: [gitlab@[MASKED].iam.gserviceaccount.com]

Finally got it, problem of variable protection, https://gitlab.com/gitlab-com/support-forum/issues/4660 gave me the lead I was missing.

Got working App Engine CI/CD with:

deploy:
  stage: deploy
  image: google/cloud-sdk:alpine
  only:
    - dev
  before_script:
    - echo $FIREBASE_KEY > ${CI_PROJECT_DIR}/key.json
    - echo $GCLOUD_SERVICE_KEY > /tmp/$CI_PIPELINE_ID.json
    - gcloud auth activate-service-account --key-file /tmp/$CI_PIPELINE_ID.json
    - gcloud config set project $CLOUDSDK_CORE_PROJECT
  script:
    - cd ${CI_PROJECT_DIR} 
    - gcloud app deploy
  allow_failure: false

And Gitlab CI/CD conf as:

@linceaerian File variable is the type of Pipeline Variable as shown below:

Content will be same. In that case your variable will hold file path of the temporary file that is autmatically created by Gitlab CI. So you will not need to manually create a file and put the content in it.

@muratcorlu, Thanks a lot, I found it later in the day, i've already switched it for the credential part :)

This was a helpful page if you are not familiar with gcloud deploy https://cloud.google.com/sdk/gcloud/reference/functions/deploy

This + the comments helped me as a template. Thank you so much.