jjamor/kong-api-gateway.md

## kong-api-gateway.md

      
    Raw
  

              kong-api-gateway.md
            
          
    Kong API Gateway


Architecture


Built on nginx + openresty
LUA as scripting language
API abstraction over nginx
Plugin architecture
Available with Docker
Configuration in Cassandra/Postgresql


Start Kong (Docker/Linux)

docker run -d --name kong-database \
              -p 5432:5432 \
              -e "POSTGRES_USER=kong" \
              -e "POSTGRES_DB=kong" \
              postgres:9.4
			  
docker run -d --name kong1 \
              --link kong-database:kong-database \
              -e "KONG_DATABASE=postgres" \
              -e "KONG_CASSANDRA_CONTACT_POINTS=kong-database" \
              -e "KONG_PG_HOST=kong-database" \
              -p 8000:8000 \
              -p 8443:8443 \
              -p 8001:8001 \
              -p 7946:7946 \
              -p 7946:7946/udp \
              kong:0.10.1
			  
curl localhost:8001
docker ps --format "table {{.ID}}\t{{.Image}}\t{{.Names}}"


Start Kong (Docker/Windows)

docker run -d --name kong-database ^
              -p 5432:5432 ^
              -e "POSTGRES_USER=kong" ^
              -e "POSTGRES_DB=kong" ^
              postgres:9.4		
			  
docker run -d --name kong1 ^
              --link kong-database:kong-database ^
              -e "KONG_DATABASE=postgres " ^
              -e "KONG_CASSANDRA_CONTACT_POINTS=kong-database" ^
              -e "KONG_PG_HOST=kong-database" ^
              -p 8000:8000 ^
              -p 8443:8443 ^
              -p 8001:8001 ^
              -p 7946:7946 ^
              -p 7946:7946/udp ^
              kong:0.10.1
			  
start http://localhost:8001
docker ps --format "table {{.ID}}^t{{.Image}}^t{{.Names}}"


Exploring the database (linux)

docker exec -it kong-database bin/bash #(or /bin/sh from windows)

$> psql -U kong -W #(pass: kong)
=# \l
=# \c kong
=# \dt
=# select * from nodes;
=# select * from apis;
=# \q


Kong CLI

docker exec -it kong1 bin/bash

$> kong version
$> kong health
$> cat /etc/kong/kong.conf.default
$> kong reload


Configuration


Any envvar starting with KONG_ will overwrite a value
Includes nginx config
proxy_port 8000 <-- traffic from clients
proxy_ssl_port 8443 <-- https
admin_api_port: 8001 <-- admin


Clustering

docker run -d --name kong2 ^
              --link kong-database:kong-database ^
              -e "KONG_DATABASE=postgres" ^
              -e "KONG_CASSANDRA_CONTACT_POINTS=kong-database" ^
              -e "KONG_PG_HOST=kong-database" ^
              kong:0.10.1


Shares the database (autodiscovering in a table)
cluster_listen address port in config
kong cluster reachability -> to check the cluster
kong quit -> to remove the cluster
kong cluster force-leave -> to force the removal


Playground box

docker run -it --rm --link kong1:kong1 --link kong2:kong2 bash
$> apk add --update python py-pip ca-certificates
$> pip install httpie httpie-unixsocket 
$> apk add jq


Creating an API

http POST kong1:8001/apis 
http kong1:8001/apis
http POST kong1:8001/apis name=demo \
	   hosts=demo.com upstream_url=http://httpbin.org
  
http kong1:8001/apis/demo
http kong2:8001/apis/demo 
http kong1:8000/headers Host:demo.com


Key authentication

http kong1:8000/headers Host:demo.com
http POST kong1:8001/apis/demo/plugins name=key-auth
http kong1:8000/headers Host:demo.com

http POST kong1:8001/consumers username=alice  \
	   custom_id=alice@wonderland.com
http POST kong1:8001/consumers/alice/key-auth key=1111
http POST kong1:8001/consumers/alice/key-auth key=2222
http kong1:8000/headers Host:demo.com apikey:1111
http DELETE kong1:8001/consumers/alice/key-auth/1111


Additional x-forwarded-* headers added
Additional x-consumir-id header forwarded
Multiple credentials can be provided for the same consumer


Anonymous users

# Check authenticated access
http head kong1:8000 Host:demo.com apikey:2222
# Check anonymous access (fails)
http head kong1:8000/headers Host:demo.com

# Get key-auth plugin id
plugin=$(http kong1:8001/apis/demo/plugins | \
	jq -r '.data[] | select(.name=="key-auth") | .id')
# Create a user and get its ID
http POST kong1:8001/consumers username=anonymous_user  
user=$(http kong1:8001/consumers/anonymous_user | jq -r ".id")
http PATCH kong1:8001/apis/demo/plugins/$plugin config.anonymous=$user

# Check again anonymous access (success)
http kong1:8000/headers Host:demo.com


JWT Authentication

http POST kong1:8001/apis/demo/plugins name=jwt config.uri_param_names=jwt
http POST kong1:8001/consumers/alice/jwt algorithm=HS256 \
	   key=alice-key-1  secret=mysecret
http kong1:8000/headers/?jwt=xxxxxxxxx (jwt token)
http DELETE kong1:8001/consumers/alice/jwt/alice-key-1


Rate limiting

# API level
http POST kong1:8001/apis/demo/plugins name=rate-limiting \
	   config.second=1 config.hour=100
# Consumer level
user=$(http kong1:8001/consumers/anonymous_user | jq -r ".id")
http POST kong1:8001/apis/demo/plugins name=rate-limiting  \ 
	   consumer_id=$user config.second=1 config.hour=100


Request size limiting


Important for preventing flood attacks
Size in MB, decimals allowed

http POST kong1:8001/apis/demo/plugins name=request-size-limiting \
	   config.allowed_payload_size=0.5


Request correlation


Very important for distributed systems
Enables traceability
Format: ip-port-nginx_pid-connection-connection_counter-timestamp

http POST kong1:8001/apis/demo/plugins \
	   name=correlation-id config.header_name=X-Kong-ID \ 
	   config.generator=tracker
http kong1:8000/headers Host:demo.com


Logs


Kong errors are in the nginx log (/usr/local/kong/logs/*)
Logging plugins are logs of forwarded request/responses
Asynchronous
Can act over specific consumers
/tmp may not be the best path

http POST kong1:8001/apis/demo/plugins name=file-log config.path=/tmp/request.log
http kong1:8000/headers Host:demo.com

docker exec -it kong1 /bin/bash
cat /tmp/request.log


Monitoring with Datadog


Create a DataDog account
Update kong instance:

docker exec -it kong1 /bin/bash

$>yum install sysstat
$>DD_API_KEY=<API_KEY> sh -c \
    "$(curl -L https://raw.githubusercontent.com/DataDog/dd-agent/master/packaging/datadog-agent/source/setup_agent.sh)"


Install datadog plugin:

http POST kong1:8001/apis/demo/plugins name=datadog \
       config.host=127.0.0.1 config.port=8125 config.timeout=1000


Generate metrics with

http kong1:8000/headers Host:demo.com


Open https://app.datadoghq.com/metric/explorer

Select "kong.demo.request.count"
Make the chart XL


Monitoring alternatives


Prometheus


Custom configuration


Not the best part of Kong

docker exec -it kong1 /bin/bash

$> vi /usr/local/share/lua/5.1/kong/templates/nginx_kong.lua


Add add_header X-Javi washere; after location / {


Cache


Uses nginx proxy_cache module
Blog post


Kong on mesos (1/4)


Official support
Launch the cluster using the portal
Stablish the tunnel with

ssh -L 80:localhost:80 -N \    
       <username>@<clusterprefix>mgmt.westeurope.cloudapp.azure.com -p 2200


Kong on mesos (2/4)


Check the portal with opening http://localhost:80
Download and install the dc-os CLI
Deploy everything!

dcos config set core.dcos_url http://localhost
dcos auth login
git clone https://github.com/<username>/kong-dist-mesos
cd kong-dist-mesos
dcos marathon app add postgres.json
dcos marathon app add kong_postgres.json
dcos marathon task list


Kong on mesos (3/4)


Check the deployment

$local> ssh-agent
# set SSH_AUTH_SOCK & SSH_AGENT_PID variables
# No, really, set them.
$local> ssh-add .ssh\id_rsa
#Jump to master
$local> ssh <username>@<clusterprefix>mgmt.westeurope.cloudapp.azure.com -A -p 2200
# check connectivity and services
$master> curl marathon-lb.marathon.mesos:10001
$master> curl marathon-lb.marathon.mesos:10002
#stablish tunnel from master:8001 to lb:10002
$master> ssh -L 8001:localhost:10002 -N marathon-lb.marathon.mesos
#stablisht unnel from local:8001 to master:8001
$local> ssh -L 8001:localhost:8001 -N <username>@<clusterprefix>mgmt.westeurope.cloudapp.azure.com -p 2200
#test local access
$local> http localhost:8001
#setup kong api
$local> http POST localhost:8001/apis name=demo \
     hosts=<clusterprefix>agents.westeurope.cloudapp.azure.com \
     upstream_url=http://httpbin.org
#jump to master again and test the API
$local> ssh <username>@<clusterprefix>mgmt.westeurope.cloudapp.azure.com -p 2200
$master> apt-get install httpie
$master> http marathon-lb.marathon.mesos:10001/headers \
     Host:<clusterprefix>agents.westeurope.cloudapp.azure.com


Kong on mesos (4/4)


Update the ACS public load balancer configuration:

Create probe probe marathon-lb-10001 to port 10001
Modify LbRuleHTTP: backend port 80 -> 10001, probe marathon-lb-10001
Add a rule to dcos-agent-public-nsg allowing traffic to 10001
Wait. Wait more.
Open http://<clusterprefix>agents.westeurope.cloudapp.azure.com


Additional tools and resources


Kong dashboard
Kongfig
Gitter channel


End of the module