Skip to content

Instantly share code, notes, and snippets.

@alexellis

alexellis/civo-workshop.md

Last active July 10, 2019 18:59
Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save alexellis/a6ee5f094f86987a0dc508442220c52a to your computer and use it in GitHub Desktop.
Save alexellis/a6ee5f094f86987a0dc508442220c52a to your computer and use it in GitHub Desktop.
Download ZIP
Civo workshop
Raw
civo-workshop.md

Civo on-site workshop with OpenFaaS Ltd

The purpose of this time is to explore OpenFaaS and to relate it to the team's existing skills around the Ruby eco-system. We will be using OpenFaaS with Kubernetes (k3s).

Can I use this too?

This workshop is available to the public under the MIT license. Anyone can follow along.

The session

Agenda:

  • brief introduction to OpenFaaS and concept of Serverless
  • provision Kubernetes using a Civo VM
  • try some of the labs
  • work through the Civo extensions for writing code in Ruby
  • consume secrets such as API tokens
  • work with HTTP headers
  • work with queued / asynchronous invocations

Throughout the exercises we'll tie back what we are learning to the team's existing skills and platform.

Get a Civo.com account

You can get 50 USD free credit via http://bit.ly/2Lx9d2o

Setup a VM

  • Setup a new Medium VM (20 USD / mo)

This comes with 2CPU 4GB RAM and 50GB disk. The Small (10 USD / mo) with 2GB RAM is also workable if you want to stretch your credit out.

  • For the Image, pick Ubuntu 18.04 LTS

  • Initial user: civo

  • Public IP Address: Create

  • Add your SSH Key

If you don't have one yet, run ssh-keygen and accept all defaults.

  • Once created, note down the Public IP:
export IP="185.136.234.31" # for example

Log into the VM with ssh

ssh civo@$IP
  • Install k3s, which provides Kubernetes

Replace 185.136.234.31 with the value in $IP:

curl -sLS get.k3s.io | INSTALL_K3S_EXEC='server --tls-san 185.136.234.31' sh -
  • Double check you used the correct IP, you should see it below:
sudo systemctl cat k3s
  • Create a Civo firewall

By default k3s is insecure because it exposes the flannel networking ports.

  • Create a firewall

Firewalls in Civo are whitelist-based, so allow:

  • 22 (SSH)
  • 80 (HTTP)
  • 443 (HTTPS)
  • 6443 (Kubernetes API)
  • 31112 (OpenFaaS Gateway)

Get your kubeconfig for kubectl

On the VM we need to make the k3s file readable:

  • mkdir ~/.kube
  • sudo cp /etc/rancher/k3s/k3s.yaml ~/.kube/config
  • sudo chown civo:civo .kube/config

Now on your laptop copy it down:

export IP="185.136.234.31" # for example
  • mkdir -p ~/dev/workshop/ && cd ~/dev/workshop
  • scp civo@$IP:~/.kube/config .

Update the localhost reference to the remote address:

  • sed -ie s/localhost/$IP/g config

Now see if kubectl is working:

export KUBECONFIG=`pwd`/config
  • kubectl get all --all-namespaces

If you open a new terminal later on, type in:

cd ~/dev/workshop
export KUBECONFIG=config

Start the workshop from lab 1b

There are some pre-requisites to install, which you may already have.

When installing OpenFaaS from the helm chart pick "A) For local clusters", this is because we do not have a LoadBalancer available when running k3s on a Civo VM.

Your gateway address will be: http://$IP:31112/ and the password is saved in gateway-password.txt

You can set it in OPENFAAS_URL, i.e.:

export OPENFAAS_URL=http://$IP:31112

This will save on typing -g or --gateway when using the faas-cli.

It's suggested that you run through at least the following before starting the extensions:

1.0 Civo extensions

1.1 The curl function and the Civo API

Deploy the curl function from the store.

Go to the Civo API docs

Copy and paste the example for how to List Instances:

-H "Authorization: bearer (redacted)" https://api.civo.com/v2/instances

Change -H to -s -H

Now copy and paste the whole command to the UI and execute it.

Did it work? What do you think?

1.2 Ruby template

There is a built-in ruby template in the templates repo.

  • Find its Dockerfile. How would you change the Ruby version?

  • Create a function and deploy it:

export HUB_USERNAME="" # Your docker hub name

faas-cli template pull

faas-cli new --lang ruby rss-tester --prefix $HUB_USERNAME

faas-cli up -f rss-tester.yml

Add a public, pure-Ruby gem to your function and consume it in the handler.rb file.

Let's use simple-rss found from awesome-ruby.

require 'rubygems'
require 'simple-rss'
require 'open-uri'
	
class Handler
  def run(req)
    uri = ENV["URI"]
    rss = SimpleRSS.parse open(uri)
    return rss.channel.title
  end
end

handler.rb

Set an environment variable:

    environment:
      URI: http://rss.slashdot.org/Slashdot/slashdot/to

rss-tester.yml

Now configure the gem needed for the build:

source 'https://rubygems.org'

gem 'simple-rss'

Gemfile

Run the following to redeploy the function

faas-cli up -f rss-tester.yml

Get its URL and invoke it:

faas-cli describe -f rss-tester.yml rss-tester

Did it work? Can you try a different RSS feed?

  • Task: take a sample from the Civo API developer guide and call one of the APIs from your handler.rb file

Note: make sure you do not hard-code any secret keys or API keys in the Ruby code, it will be uploaded to the Docker Hub

In 2.0 we'll see how to use a secure Kubernetes secret, for this exercise use an environment variable in your YAML file.

You can read environment variables in Ruby with ENV["name-of-variable"]

For example, how about listing all VM templates?

Create a new function:

export HUB_USERNAME="" # Your docker hub name

faas-cli template pull

faas-cli new --lang ruby civo-api-tester --prefix $HUB_USERNAME

Add an environment variable:

    environment:
      api-token: x-z-y

Deploy it, then iterate on the code:

faas-cli up -f civo-api-tester.yml

Remember to run faas-cli up -f civo-api-tester.yml after each change to the Ruby code or Gemfile.

Links: https://www.civo.com/api

1.3 Ruby HTTP template

There is another Ruby template that provides greater control over HTTP responses.

View the Dockerfile and how it's different from the original Ruby template:

https://github.com/openfaas-incubator/ruby-http

  • What are the key differences?

Please read: Templates in OpenFaaS - classic vs. of-watchdog

  • Create a new function using the ruby-http template:
export HUB_USERNAME="" # Your docker hub name

faas-cli template store list


faas-cli template store pull ruby-http

faas-cli new --lang ruby-http user-test --prefix $HUB_USERNAME

faas-cli up -f user-test.yml

Write code in handler.rb to read the X-User-Id header. Return a header "X-User-Status" with value of OK or Not Found.

curl -i http://$IP:31112/function/user-test -H "X-User-Id: mark`
X-User-Status: OK

curl -i http://$IP:31112/function/user-test -H "X-User-Id: john`
X-User-Status: Not Found

1.4 Try compiling a native module: nokogiri

Try the following example:

https://github.com/alexellis/openfaas-nokogiri

Once you have it working, think of another public gem which needs compilation and try that out with the same approach.

2.0 Event-driven functions and the Civo API

The Civo API publishes events in the form of webhooks every time something happens in your account.

  • Create a new Ruby HTTP function:
export HUB_USERNAME="" # Your docker hub name

faas-cli template store list

faas-cli template store pull ruby-http

faas-cli new --lang ruby-http civo-hooks --prefix $HUB_USERNAME

faas-cli up -f civo-hooks.yml
  • Now go to your Settings page and click Webhooks

Get the public endpoint for your new function:

faas-cli describe -f civo-hooks.yml civo-hooks

Let's use the async route so that our webhook can return immediately, and our function will be invoked when capacity is available in the cluster.

  • Now click Create Webhook

  • Click All Events

  • Enter URL of the value received in faas-cli describe

  • Click Create Webhook

Note down the Secret that will be used to generate a hash for each message you receive.

It's time to trigger the webhook

  • Start tailing the logs of the queue-worker:
kubectl logs -n openfaas deploy/queue-worker -f

  • Create a extra-small instance

  • Check to see if your function is invoked

The queue-worker will show the body that was used to invoke the function including any headers.

Can you see the following?

  • X-Civo-Signature

  • X-Civo-Event

  • User-Agent

  • Check the logs of the function:

kubectl logs -n openfaas-fn deploy/civo-hooks

  • How do we know if the person calling our endpoint was really the Civo API and not a bad actor?

  • Create a secret for the webhoook secret

echo -n "webhook-secret-from-settings-page" | faas-cli secret create civo-webhook
  • Bind the secret to the function and deploy it again

Under this section:

version: 1.0
provider:
  name: openfaas
  gateway: http://127.0.0.1:8080
functions:
  civo-hooks:

Add:

    secrets:
    - civo-webhook

faas-cli up -f civo-hooks.yml
  • Now that your function has the secret, you can use HMAC to validate the incoming body

This is a stretch goal and will require some coding.

Andy has written up a page on how to validate webhooks in Ruby, you may need some additional gems.

Try it below:

https://www.civo.com/learn/using-webhooks-to-receive-important-change-callbacks

Use this to get started:

require 'openssl'

class Handler
  def run(body, headers)
    response_headers = {"content-type": "text/plain"}
    secret_value = "hard-coded"

    signature = OpenSSL::HMAC.hexdigest(
     OpenSSL::Digest.new('sha1'),
     secret_value,
     body
   )

    return signature, response_headers
  end
end

handler.rb

source 'https://rubygems.org'

gem 'openssl'

Gemfile

  • Can you now read the secret file we created called civo-webhook?

Find out how in the docs: secrets

  • Finally use the secret to validate the payload and return the appropriate HTTP code.

Wrapping up

For further information see: the OpenFaaS blog and Slack community

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment