Create Cloud Run Service through Deployment Manager

cloudrun.jinja

{% set PROJECT = env['project'] %}
{% set REGION = properties['region'] | default('us-east1') %}

{% set PARENT = 'projects/' + PROJECT + '/locations/' + REGION %}

{% set PREFIX = env['deployment'] %}

{% set CLOUDRUN_TYPE_PROVIDER = PREFIX + '-provider' %}
{% set CLOUDRUN_SERVICE_TYPE = PROJECT + '/' + CLOUDRUN_TYPE_PROVIDER + ':projects.locations.services' %}

{% set CLOUDRUN_SERVICE = PREFIX + '-service' %}
{% set CLOUDRUN_SERVICE_ACCOUNT = PREFIX + '-sa' %}

{% set CLOUDRUN_SERVICE_INVOKER = PREFIX + '-service' %}
{% set CLOUDRUN_INVOKER = properties['invoker'] | default('allUsers') %}

resources:
- name: {{ CLOUDRUN_TYPE_PROVIDER }}
  type: deploymentmanager.v2beta.typeProvider
  properties:
    descriptorUrl: https://run.googleapis.com/$discovery/rest?version=v1alpha1
    options:
      inputMappings:
      - fieldName: Authorization
        location: HEADER
        value: $.concat("Bearer ", $.googleOauth2AccessToken())

    collectionOverrides:
    - collection: projects.locations.services
      options:
        virtualProperties: |
          schema: http://json-schema.org/draft-04/schema#
          type: object
          required:
          - spec
          properties:
            spec:
              type: object
              description: https://cloud.google.com/run/docs/reference/rest/v1alpha1/RevisionSpec

        inputMappings:
        - methodMatch: ^create$
          location: PATH
          fieldName: parent
          value: $.concat("{{ PARENT }}")

        - methodMatch: ^(get|replaceService|delete)$
          location: PATH
          fieldName: name
          value: $.concat("{{ PARENT }}/services/", $.resource.name)

        - methodMatch: ^setIamPolicy$
          location: PATH
          fieldName: resource
          value: $.concat("{{ PARENT }}/services/", $.resource.name)

        - methodMatch: ^create$
          location: BODY
          fieldName: kind
          value: $.concat("Service")

        - methodMatch: ^create$
          location: BODY
          fieldName: apiVersion
          value: $.concat("serving.knative.dev/v1alpha1")

        - methodMatch: ^create$
          location: BODY
          fieldName: metadata.name
          value: $.resource.name

        - methodMatch: ^replaceService$
          location: BODY
          fieldName: metadata
          value: $.resource.self.metadata

        - methodMatch: ^(create|replaceService)$
          location: BODY
          fieldName: spec.template.spec
          value: $.resource.properties.spec

- name: {{ CLOUDRUN_SERVICE }}
  type: {{ CLOUDRUN_SERVICE_TYPE }}
  metadata:
    dependsOn:
    - {{ CLOUDRUN_TYPE_PROVIDER }}
  properties:
    spec:
      serviceAccountName: $(ref.{{ CLOUDRUN_SERVICE_ACCOUNT }}.email)
      containerConcurrency: 5
      timeoutSeconds: 10
      containers:
      - image: marketplace.gcr.io/google/nginx1
        env:
        - name: NGINX_HOST
          value: example.com
        resources:
          limits:
            memory: 128Mi
  accessControl:
    gcpIamPolicy:
      bindings:
      - role: roles/run.invoker
        members:
        - {{ CLOUDRUN_INVOKER }}

- name: {{ CLOUDRUN_SERVICE_ACCOUNT }}
  type: iam.v1.serviceAccount
  properties:
    accountId: {{ CLOUDRUN_SERVICE_ACCOUNT }}
    displayName: Service account for {{ CLOUDRUN_SERVICE }}

This config enables deployment of a Google Cloud Run service through Deployment Manager.

Unfortunately, updates to Cloud Run Service resource don't work, which appears to be because Deployment Manager expects all PATH parameters to be named the same between POST and PUT methods, however the Cloud Run API v1alpha1 defines them as

{
  "id": "run.projects.locations.services.create",
  "httpMethod": "POST",
  "path": "v1alpha1/{+parent}/services",
  "flatPath": "v1alpha1/projects/{projectsId}/locations/{locationsId}/services"
},
{
  "id": "run.projects.locations.services.replaceService",
  "httpMethod": "PUT",
  "path": "v1alpha1/{+name}",
  "flatPath": "v1alpha1/projects/{projectsId}/locations/{locationsId}/services/{servicesId}"
}

so in POST we have parent parameter, while in PUT it's name. DM docs appear to prohibit such combinations (with no way around it):

Every path/query parameter for a POST must be a parameter for PUT
The following is invalid because myParameter exists for POST but not for PUT:

POST  /my-api/{myParameter}/resource/{mySecondParameter}
PUT   /my-api/resource/{mySecondParameter}  # myParameter is not present

So unless Cloud Run API v1/v1beta1 renames parent to name (or vice versa), we'll still be out of luck..

The DeploymentManager team is actively working on the native support for the CloudRun API where this issue with the update parameters will be mitigated.

As a respect of your great effort to write this type-provider I reached out the DM team and we found an experimental solution for you.

Note: type: deploymentmanager.local.typeProvider is experimental, undocumented, alpha.

I tested this custom type provider manually, I managed to CRUD cloudrun services with it.

- name: {{ CLOUDRUN_TYPE_PROVIDER }}
  type: deploymentmanager.local.typeProvider
  properties:
    descriptorUrl: https://run.googleapis.com/$discovery/rest?version=v1alpha1
    options:
      inputMappings:
      - fieldName: Authorization
        location: HEADER
        value: $.concat("Bearer ", $.googleOauth2AccessToken())

    collectionOverrides:
    - collection: projects.locations.services
      methodMap:
        get: run.projects.locations.services.get
        create: run.projects.locations.services.create
        delete: run.projects.locations.services.delete
        update: run.projects.locations.services.replaceService
        setIamPolicy: run.projects.locations.services.setIamPolicy
      options:
        virtualProperties: |
          schema: http://json-schema.org/draft-04/schema#
          type: object
          properties:
            spec:
              type: object
              description: https://cloud.google.com/run/docs/reference/rest/v1alpha1/RevisionSpec

        inputMappings:
        - methodMatch: ^create$
          location: PATH
          fieldName: parent
          value: $.concat("{{ PARENT }}")

        - methodMatch: ^(get|replaceService|delete)$
          location: PATH
          fieldName: name
          value: $.concat("{{ PARENT }}/services/", $.resource.name)

        - methodMatch: ^setIamPolicy$
          location: PATH
          fieldName: resource
          value: $.concat("{{ PARENT }}/services/", $.resource.name)

        - methodMatch: ^(create|replaceService)$
          location: BODY
          fieldName: kind
          value: $.concat("Service")

        - methodMatch: ^(create|replaceService)$
          location: BODY
          fieldName: apiVersion
          value: $.concat("serving.knative.dev/v1alpha1")

        - methodMatch: ^create$
          location: BODY
          fieldName: metadata.name
          value: $.resource.name

        - methodMatch: ^replaceService$
          location: BODY
          fieldName: metadata
          value: $.resource.self.metadata

        - methodMatch: ^(create|replaceService)$
          location: BODY
          fieldName: spec.template.spec
          value: $.resource.properties.spec

Excellent, I'll add this to our "explore next" list! Since TF doesn't support Cloud Run IAM policies just yet, DM seems to be the closest in terms of full support atm.

I have 2 updates for you:

• The CloudRun V1 API went to GA a few days ago. The api structure is the same as the V1Alpha1 which you used, excet...
• The projects.locations.services and projects.locations.domainmappings resources are deprecated, I advise you to use the v1.namespaces.services and v1.namespaces.domainmappings resources.

Native support for gcp-types/run-v1 is under development.

Perfect, thanks!

Have there been any DM changes re Cloud Run GA announcement today?

The DM team is actively working on the GA type provier but we are facing minor issues with the API (because of the multi-region endpoints). This should be resolved soon.

Thanks for the update!

Are there any updates in this direction (support for Cloud Run through Deployment Manager)?

@ocsig and @dinvlad Is there any python template available for cloudrun deployment using google cloud deployment manager?

Hey @ocsig - great info! is there any further update on the native support?

I am having one more issue. Would appreciate some help here as I am stuck.

Please refer my issue
https://stackoverflow.com/questions/67065062/create-cloud-run-service-with-deployment-manager/68379784#68379784

Here is a summary:

Despite having the authorization header in the options inputMappings, I am getting a 401

ERROR: (gcloud.beta.deployment-manager.deployments.create) Error in Operation [operation-1626271807378-5c715e9631374-01345f3b-a23ceb5e]: errors:
- code: RESOURCE_ERROR
  location: /deployments/my-deployment-001/resources/my-service-001
  message: '{"ResourceType":"spatial-cat-319209/tp-cloudrun:run.projects.locations.services.create","ResourceErrorCode":"401","ResourceErrorMessage":{"code":401,"message":"Request
    is missing required authentication credential. Expected OAuth 2 access token,
    login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.","status":"UNAUTHENTICATED","statusMessage":"Unauthorized","requestPath":"https://run.googleapis.com/v1/projects/spatial-cat-319209/locations/asia-south1/services","httpMethod":"POST"}}'

Hello @VikramVasudevan - I'm not sure if there's native support or an API available for cloud run. I checked through this link and none exists. I think one still needs to go down the route of declaring a custom base type for cloud run.

Thanks for confirming @Bashorun97 - any advice on my issue though?

@VikramVasudevan I'd really recommend using Terraform, tbh. Can't help much more since we stopped using DM for this and other reasons. Terraform is much more flexible and logical (and easier to troubleshoot)!

Thanks for writing back @dinvlad
I have to say I am very surprised google's own sdk is not good enough (atleast not yet).
Will explore terraform.

From almost 3 years ago The DM team is actively working on the GA type provier but we are facing minor issues with the API (because of the multi-region endpoints). This should be resolved soon.

The next time I meet with our google reps I'll have to ask them about the Deployment Manager road map because it really seems like its been abandoned. Its especially frustrating to not have support for cloud run because its serverless. serverless stuff is like the easiest use case for IaC but I have to roll my own janky scripts to deploy it- terraform isnt great for deploying apps

Yep, although it's possible to do Docker even in Terraform:
https://registry.terraform.io/providers/kreuzwerker/docker/latest/docs/resources/image#build
(but we're actually using https://github.com/GoogleContainerTools/kaniko for it via separate Cloud Build steps, which works nicely).