Graphcool-AWS integration ideas

_Integration ideas.md

Graphcool and AWS functions

Context

I have a Graphcool project definition, with multiple stages, and I have a Serverless AWS definition. I use the Serverless AWS definition to deploy all of my functions, and I link them to Graphcool by using Webhooks. This document describes some of the ideas for linking those two deployments together.

Information exchange

So what kind of information do the Graphcool service and the Serverless service need to know about each other in order to connect?

• If I have a function that I reference in my Graphcool service, Graphcool needs to know the URL of that function on AWS Lambda
• If I have a function on AWS Lambda only that connects to Graphcool (so an external endpoint, a cron job, anything that's not called from Graphcool), it needs to know the URL of my Graphcool endpoint.

Basic implementation

The easiest way to implement this, is based on the following assumptions:

• Graphcool and Serverless use the same stage name to deploy
• If Graphcool uses a local cluster, so should Serverless

Advanced implementation

This basic implementation can be extended, using the following assumptions:

• When deploying, the Graphcool and Serverless stages are both specified
• For Serverless, it is specified if it should run local or not

Basic implementation details

To run the AWS API Gateway locally, we use the new serverless platform beta (https://serverless.com/framework/docs/platform/). It runs on localhost, and you can specify the port is runs on. It also supports debugging (using VS Code or Chrome).

So how can we get the information exchange set up? These are the steps the new command should take for a local stage:

• You start our command, and specify a Graphcool stage name
• Get the cluster and serviceId from .graphcoolrc
• If cluster is not one of the shared ones, look up the url in ~/.graphcoolrc
• Construct the simple API endpoint address (http://${clusterUrl}/simple/v1/${serviceId})
• Construct the AWS API base URL. This will be http://localhost:4000/${functionPath}
• Store the AWS API base URL in an environment variable
• The script will run graphcool local up and sls run

These are the steps the new command should take for a 'normal' stage:

• You start the script, and specify a Graphcool stage name
• Get the cluster and serviceId from .graphcoolrc
• Construct the simple API endpoint address (https:/api.graph.cool/simple/v1/${serviceId})
• Store the Graphcool endpoint address in an environment variable
• Construct the AWS API base URL. The first time you deploy to a stage, AWS generates a random prefix. After that, you can use: "https://XXXXXXXXX.execute-api.${region}.amazonaws.com/${stage}/${functionPath}"
• Store the AWS API base URL in an environment variable
• The script will run graphcool deploy and sls deploy

In both cases:

• Use the AWS base URL env var in graphcool.yml for all webhooks. Specify path is you use path in serverless.yml
• Use the Graphcool base URL as a variable you pass to your functions in serverless.yml

Advanced implementation details

If you want to connect local and cloud stages (for either Graphcool or AWS), you could do the following:

• The script allows you to pass in graphcool stage and aws stage seperately
• if graphcool stage == local and aws stage != local -> run ngrok pointing to graphcool and use that as graphcool url for the env var (tested and working on Linux)
• if graphcool stage != local and aws stage == local -> run ngrok pointing to aws and use that as url for the aws base url env var (also tested and working on Linux)

TODO

• It might be possible to get the exact URL from serverless by running a small plugin, that sets the env var when deploy is done. This allows for a bit more flexibility, for example when passing a region
• It would be nice to pass the region of the Graphcool shared cluster to sls to make sure your functions automatically run in the same region as your Graphcool endpoint
• Another very interesting development is AWS SAM. With AWS SAM and aws-sam-local, you can achieve the same thing as with sls. Also, the aws-sam-local environment is far superior over anything SLS has (including serverless-offline -> bad because it uses your local node version and serverless-simulate-plugin -> better, but debugging is not supported). With AWS SAM, the rest of the script stays exactly the same.
• If I need a rootToken in an AWS function, I might be able to use an environment variable with the name of the rootToken in my serverless.yml for the function, and call graphcool root-token for every rootToken in my graphcool.yml between the graphcool deploy and the sls deploy, setting those environment variables.
• It might be possible to use the AWS CLI/SDK to retrieve the information about the api gateway generated id, and the endpoints for the functions (not looked into it yet). Update: it is:
• There's a circular dependency between Graphcool and AWS, and that can't be solved by a single deployment:
  • Graphcool rootTokens are only known after deployment
  • Graphcool serviceId is only know after deployment
  • AWS API ids (the XXXXXX in the URL) are only known after first deployment to a stage/region combo
• If the script calls the deploy, instead of environment variables, setting process.env[....] will also work, because the deploys are called as child processes
• Update on circular dep: if you use AWS SSM to store the variables, then you can deploy them independent of deploying the functions. Also, you don't have to use environment variables, because you store them directly in the SSM store in our script.

graphcool.yml

# sample function definition with env variables
functions:
  hello:
    handler:
      webhook:
        url: {env:AWS_BASE_URL}/hello
    type: resolver
    schema: ./src/hello.graphql
    
rootTokens:
- someToken

serverless.yml

# Sample serverless.yml
# hello is called from Graphcool, so it gets endpoints and token automatically
# somefunction is an external endpoint, that needs endpoint and token manually

functions:
  hello:
    handler: handler.hello
    events:
      - http:
          path: hello
          method: get
  
  somefunction:
    handler: handler.somefunction
    events:
      - http:
          path: somefunction
          method: post
    environment:
      GRAPHCOOL_SIMPLE_ENDPOINT: {env:GRAPHCOOL_SIMPLE_ENDPOINT}
      GRAPHCOOL_ROOT_TOKEN: {env:TOKEN_someToken}