Skip to content

Instantly share code, notes, and snippets.

@olivernadj

olivernadj/go-lab2.md

Last active November 6, 2019 09:47
Show Gist options
  • Save olivernadj/d78dd8fbd0d1e0a2f597d3c457165f70 to your computer and use it in GitHub Desktop.
Save olivernadj/d78dd8fbd0d1e0a2f597d3c457165f70 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
go-lab2.md
tags
Golang, Swagger, Prometheus, Grafana

Week 2.2 Lab: High performance API

During this lab we will generate Go API from swagger and build a monitoring services around. Thanks to monitoring we can measure the performance of our API in real time.

Pre-request

You need an working docker-compose orchestration on your working laptop.

And following docker images. Please pull them before you go to class. To avoid network issues.

$ docker pull swaggerapi/swagger-editor
$ docker pull golang:1.12
$ docker pull prom/prometheus:v2.8.0
$ docker pull grafana/grafana:6.0.2

Good practice when you play with dockers

Whenever I am working with/on dockers I always watching running containers in a separated terminal.

$ watch "sudo docker ps --format='table{{.Image}}\t{{.Names}}\t{{.Status}}\t{{.Ports}}'"

Milestone 1: Generate your API from swagger

Start the swagger editor

docker pull swaggerapi/swagger-editor
docker run -d -p 8888:8080 swaggerapi/swagger-editor

Open http://localhost:8888/ in your browser.

Pick an example of yaml specificaiton from OAI/OpenAPI-Specification. Your choice, I go with uber.yaml. Download and past the content to your swagger editor.

Yet again, feel free to modify, or create your own specification. Just don't wast much time. We need to hurry.

Download go-server from Generate server main menu.

Now you should have a go-server-server-generated.zip in your downloads.

Create project folder

Create a new project folder. Eg.: ~/go/src/github.com/olivernadj/go-lab2-uber. I will refer this as project folder.

Create goapi/src inside your project folder.

Extract the content of go-server-server folder into goapi/src

Check with it. You shuld have the something like this as:

go-lab2-uber$ tree
.
└── goapi
    └── src
        ├── api
        │   └── swagger.yaml
        ├── go
        │   ├── api_estimates.go
        │   ├── api_products.go
        │   ├── api_user.go
        │   ├── logger.go
        │   ├── model_activities.go
        │   ├── model_activity.go
        │   ├── model_error.go
        │   ├── model_price_estimate.go
        │   ├── model_product.go
        │   ├── model_product_list.go
        │   ├── model_profile.go
        │   ├── README.md
        │   └── routers.go
        └── main.go

Go to goapi/src folder and:

  • Fix the import in main.go to match with fully-qualified import path.
  • Run the server with go run main.go
  • Check the result in your browser: http://localhost:8080/v1/ you will see hello world if all good.

Yayy. Let's continue.

Stop your server.

Downloading SwaggerUI files

We just need the dist folder from SwaggerUI repo. Do not clone the entire SwaggerUI repo inside your project folder. Clone or download some temporary place.

SwaggerUI can be downloaded from their GitHub Repo. Once downloaded, place the content of dist folder in your goapi/src and rename it to swaggerui.

In your swagger editor ( http://localhost:8888/ ) download swagger.json File->Convert and save as json menu. After that, move swagger.json file to swaggerui folder, and inside index.html change url to ./swagger.json (url: "./swagger.json").

The routing is missing, but we will add that later when we will working on routers.go

Add Prometheus instrumentation

Place monitoring.go file into goapi/src/go folder with this content:

package swagger

import (
	"github.com/prometheus/client_golang/prometheus"
	"net/http"
	"strconv"
	"time"
)

func BuildSummaryVec(metricName string, metricHelp string) *prometheus.SummaryVec {
	summaryVec := prometheus.NewSummaryVec(
		prometheus.SummaryOpts{
			Name:      metricName,
			Help:      metricHelp,
			ConstLabels: prometheus.Labels{"service":"uber_api"},
			Objectives: map[float64]float64{0.5: 0.05, 0.9: 0.01, 0.99: 0.001},
		},
		[]string{"handler", "code"},
	)
	prometheus.Register(summaryVec)
	return summaryVec
}

// WithMonitoring optionally adds a middleware that stores request duration and response size into the supplied
// summaryVec
func WithMonitoring(next http.Handler, route Route, summary *prometheus.SummaryVec) http.Handler {
	return http.HandlerFunc(func(rw http.ResponseWriter, req *http.Request) {
		start := time.Now()
		lrw := NewMonitoringResponseWriter(rw)
		next.ServeHTTP(lrw, req)
		statusCode := lrw.statusCode
		duration := time.Since(start)

		// Store duration of request
		summary.WithLabelValues(route.Name, strconv.FormatInt(int64(statusCode), 10)).Observe(duration.Seconds() * 1000)
	})
}

type monitoringResponseWriter struct {
	http.ResponseWriter
	statusCode int
}

func NewMonitoringResponseWriter(w http.ResponseWriter) *monitoringResponseWriter {
	// WriteHeader(int) is not called if our response implicitly returns 200 OK, so
	// we default to that status code.
	return &monitoringResponseWriter{w, http.StatusOK}
}

func (lrw *monitoringResponseWriter) WriteHeader(code int) {
	lrw.statusCode = code
	lrw.ResponseWriter.WriteHeader(code)
}

Refactor routers.go

In your goapi/src/go/routers.go append Route struct with Monitor bool. Like:

type Route struct {
	Name        string
	Method      string
	Pattern     string
	HandlerFunc http.HandlerFunc
	Monitor     bool
}

In the bottom of the file fix Route-s by adding true or false if you don't want to monitor a route.

Add a new HandlerFunc

func Metrics(w http.ResponseWriter, r *http.Request) {
	p := promhttp.Handler()
	p.ServeHTTP(w, r)
}

And a new Route

	Route{
		"Metrics",
		strings.ToUpper("Get"),
		"/metrics",
		Metrics,
		false,
	},

Last, but not least modify your func NewRouter() what should looks like this:

func NewRouter() *mux.Router {
	router := mux.NewRouter().StrictSlash(true)
	sh := http.StripPrefix("/v1/ui/", http.FileServer(http.Dir("./swaggerui/")))
	router.PathPrefix("/v1/ui/").Handler(sh)
	summaryVec := BuildSummaryVec("http_response_time_milliseconds", "Latency Percentiles in Milliseconds")
	for _, route := range routes {
		var handler http.Handler
		handler = route.HandlerFunc
		handler = Logger(handler, route.Name)
		if route.Monitor {
			handler = WithMonitoring(handler, route, summaryVec)
		}
		router.
			Methods(route.Method).
			Path(route.Pattern).
			Name(route.Name).
			Handler(handler)
	}

	return router
}

Test Prometheus instrumentation and swagger documentation

In goapi/src folder and:

Yayy. Let's continue.

Stop your server.

Milestone 2: Pack the dependences with govendor

Install or update govendor

$ go get -u github.com/kardianos/govendor That command downloads or updates the package containing the tool, placing the source code in $GOPATH/src/github.com/kardianos/govendor and then compiles the package, placing the govendor binary in $GOPATh/bin.

Setup your project and existing GOPATH files to vendor

Go to goapi/src folder $ govendor init

Add existing GOPATH files to vendor. $ govendor add +external

View your work. $ govendor list

Milestone 3: Configure your Prometheus

Create following tree in your project folder

go-lab2-uber$ tree prometheus/

prometheus/
├── config
│   └── prometheus.yml
└── Dockerfile

The prometheus.yml should contains this:

global:
  scrape_interval:     5s # Set the scrape interval to every 5 seconds. Default is every 1 minute.
  evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.
  # scrape_timeout is set to the global default (10s).

scrape_configs:
  - job_name: uber_api
    metrics_path: "/metrics"
    static_configs:
      - targets:
          - "goapi:8080"

The Dockerfile should contains this:

FROM prom/prometheus:v2.8.0
CMD        [ "--config.file=/etc/prometheus/prometheus.yml", \
             "--storage.tsdb.path=/prometheus", \
             "--storage.tsdb.retention.time=1780d", \
             "--web.console.libraries=/etc/prometheus/console_libraries", \
             "--web.console.templates=/etc/prometheus/consoles" ]

That's it.

Milestone 4: Set up docker compose environment

Add this docker-compose.yml into your project root. Please modify olivernadj/go-lab2-uber to match with your env.

version: '3'
services:
  goapi:
    image: golang:1.12
    volumes:
      - ./goapi/src:/go/src/goapi
      - ./goapi/src:/go/src/github.com/olivernadj/go-lab2-uber/goapi/src
    working_dir: /go/src/goapi
    ports:
      - "8080:8080"
    dns: 8.8.8.8
    command: go run main.go -p 8080
    restart: always
  prometheus:
    build: ./prometheus
    volumes:
      - ./prometheus/config:/etc/prometheus
      - /prometheus
    ports:
      - "9090:9090"
    links:
      - goapi
    restart: always
  grafana:
    image: olivernadj/secret-api-grafana
    environment:
      GF_SECURITY_ADMIN_PASSWORD: 5ecret
    volumes:
      - /var/lib/grafana
    ports:
      - 3000:3000
    links:
      - prometheus
    restart: always

Now your project folder should like this:

go-lab2-uber$ tree
.
├── docker-compose.yml
├── goapi
│   └── src
│       ├── api
│       │   └── swagger.yaml
│       ├── go
│       │   ├── api_*.go
│       │   ├── logger.go
│       │   ├── model_*.go
│       │   ├── monitoring.go
│       │   ├── README.md
│       │   └── routers.go
│       ├── main.go
│       ├── swaggerui
│       │   ├── swagger.json ... and lots of files
│       │   :
│       └── vendor
│           ├── lots of files and folders....
│           :
└── prometheus
    ├── config
    │   └── prometheus.yml
    └── Dockerfile

Ths api_*.go and model_* are vary depends on your yaml file.

Milestone 5: Build and run docker compose

In your project folder build and run docker-compose

go-lab2-uber$ docker-compose up --build

Initially you should receive lot's of logs.

Milestone 6: Make a fake load against your service.

Create a fakeload.sh bash script and make it executable chmod +x ./fakeload.sh

#!/usr/bin/env bash

################################################################
# Script name      : fakeload.sh                               #
# Description      : Makes randomized apache bench requests    #
# Original Author  : Oliver Nadj <mr.oliver.nadj@gmail.com>    #
################################################################
if [[ -z "$1" ]]; then HOST="http://localhost:8080/v1"; else HOST=$1; fi
loadGet () {
    if [[ -z "$2" ]]; then SLEEP_RAND=5; else SLEEP_RAND=$2; fi
    if [[ -z "$3" ]]; then REQUESTS_RAND=5; else REQUESTS_RAND=$3; fi
    (
        while [[ 1 ]]   # Endless loop.
        do
           sleep $(( $RANDOM % $SLEEP_RAND + 1 ))
           REQUESTS=$(( $RANDOM % REQUESTS_RAND + 1 ))
           echo "ab -n ${REQUESTS} $1"
           `ab -n ${REQUESTS} $1 > /dev/null`
        done
    ) &
}
loadPost () {
    if [[ -z "$2" ]]; then SLEEP_RAND=5; else SLEEP_RAND=$2; fi
    if [[ -z "$3" ]]; then REQUESTS_RAND=5; else REQUESTS_RAND=$3; fi
    (
        while [[ 1 ]]   # Endless loop.
        do
           sleep $(( $RANDOM % $SLEEP_RAND + 1 ))
           REQUESTS=$(( $RANDOM % REQUESTS_RAND + 1 ))
           echo "ab -p fakeload-post.txt -T application/x-www-form-urlencoded -n ${REQUESTS} $1"
           `ab -p fakeload-post.txt -T application/x-www-form-urlencoded -n ${REQUESTS} $1> /dev/null`
        done
    ) &
}
trap ctrl_c INT
function ctrl_c() {
        echo "Bye"
        pkill -P $$
        exit 1
}
##### modify below this line

loadGet "$HOST/products?latitude=10.762259%26longitude=106.707844" 5 50
loadGet "$HOST/estimates/price?start_latitude=10.762259%26start_longitude=106.707844%26end_latitude=10.762259%26end_longitude=106.707844" 11 20
loadGet "$HOST/me" 3 100

##### and above this one :)
while [[ 1 ]]
do
    sleep 1
done

Endpoints of the services

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment