This guide contains instructions on manually setting up Openwhisk and CouchDB on a fresh Ubuntu 16.04 server.
The guide is based on the OpenWhisk Ansible README, which at the time of writing is missing some key steps and gotchas - hence this new guide.
sudo apt-get update
sudo apt-get upgrade
sudo apt-get install git python-pip python-setuptools build-essential libssl-dev libffi-dev python-dev software-properties-common
sudo pip install --upgrade pip
sudo pip install ansible==2.4.1.0
Install Docker:
sudo apt-key adv --keyserver hkp://p80.pool.sks-keyservers.net:80 --recv-keys 58118E89F3A912897C070ADBF76221572C52609D
sudo apt-add-repository 'deb https://apt.dockerproject.org/repo ubuntu-xenial main'
sudo apt-get install docker-engine -y
sudo usermod -aG docker $USER
$ docker -v
Docker version 17.05.0-ce, build 89658be
//enable remote api, http proxy
sudo mkdir /etc/systemd/system/docker.service.d
cat > /etc/systemd/system/docker.service.d/remote-api.conf <<EOF
[Service]
Environment="HTTP_PROXY=http://127.0.0.1:8118/"
Environment="NO_PROXY=localhost,127.0.0.1"
ExecStart=
ExecStart=/usr/bin/dockerd --debug -H tcp://0.0.0.0:2375 -H unix:///var/run/docker.sock
EOF
sudo systemctl daemon-reload
sudo service docker restart
//view dockerd log
sudo journalctl -u docker.service -f | grep Calling
You can check whether Docker is running via sudo service docker status. Also, importantly, make sure to enable the remote API.
We're using a self-hosted CouchDB here, but you can choose to skip this step and use an ephemeral CouchDB or Cloudant instead.
sudo add-apt-repository ppa:couchdb/stable -y
sudo apt-get update
sudo apt-get install couchdb -y
Secure the database and add an admin user:
sudo service couchdb stop
sudo chown -R couchdb:couchdb /usr/share/couchdb /etc/couchdb /usr/bin/couchdb
sudo chmod -R 0770 /usr/share/couchdb /etc/couchdb /usr/bin/couchdb
sudo service couchdb start
curl -X PUT localhost:5984/_config/admins/USERNAME -d '"PASSWORD"'
Set the reduce_limit property to false (as per OpenWhisk instructions):
curl -X PUT localhost:5984/_config/query_server_config/reduce_limit -d '"false"' -u USERNAME:PASSWORD
Also set bind_address=0.0.0.0 in /etc/couchdb/local.ini and restart via sudo service couchdb restart to access CouchDB from Docker containers: running curl <your CouchDB IP>:5984 should now output CouchDB info.
Important! Run the following Ansible commands with the root user (e.g. via sudo su -), since using another username seems to mess up the database table names!
git clone https://github.com/openwhisk/openwhisk.git
cd openwhisk
(cd tools/ubuntu-setup && ./all.sh)
===>
installing basics: ntp, git, zip
installing python dependences: pip, argcomplete, couchdb
installing java: oracle-java8
install scala
installing docker ====》 since docker was installed above, the docker install script can be disabled here
installing ansible
Configure our database parameters:
cd ansible
export OW_DB=CouchDB
export OW_DB_PROTOCOL=http
export OW_DB_HOST=<the IP of your CouchDB server (should be accessible from Docker containers)>
export OW_DB_PORT=5984
export OW_DB_USERNAME=USERNAME
export OW_DB_PASSWORD=PASSWORD
//test access couchdb
$ curl $OW_DB_HOST:$OW_DB_PORT
{"couchdb":"Welcome","uuid":"44460abe48f7a3eaf21af8fbf8363093","version":"1.6.1","vendor":{"name":"Ubuntu","version":"16.04"}}
ansible-playbook setup.yml
Build
Now db_local.ini should contain the correct DB settings. Let's continue with building OpenWhisk (it'll take a while):
ansible-playbook prereq.yml
cd ~/openwhisk
./gradlew distDocker
====>
BUILD SUCCESSFUL in ?????s
44 actionable tasks: 31 executed, 13 up-to-date
Deploy
And deploying it:
cd ansible
ansible-playbook initdb.yml
ansible-playbook wipe.yml
ansible-playbook apigateway.yml
ansible-playbook openwhisk.yml
ansible-playbook postdeploy.yml
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
0149a3f5f33f nginx:1.11 "nginx -g 'daemon ..." About an hour ago Up About an hour 0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp, 0.0.0.0:8443->8443/tcp nginx
8b63ba543d70 whisk/nodejs6action:latest "/bin/sh -c 'node ..." About an hour ago Up About an hour wsk1_5_prewarm_nodejs6
960f9c3099e4 whisk/nodejs6action:latest "/bin/sh -c 'node ..." About an hour ago Up About an hour wsk0_5_prewarm_nodejs6
c02926fc8604 whisk/nodejs6action:latest "/bin/sh -c 'node ..." About an hour ago Up About an hour wsk1_3_prewarm_nodejs6
ef7f2d5e6947 whisk/nodejs6action:latest "/bin/sh -c 'node ..." About an hour ago Up About an hour wsk0_4_prewarm_nodejs6
df1c90bb12b2 whisk/invoker:latest "/bin/sh -c 'exec ..." About an hour ago Up About an hour 0.0.0.0:12002->8080/tcp invoker1
0a48e32adb16 whisk/invoker:latest "/bin/sh -c 'exec ..." About an hour ago Up About an hour 0.0.0.0:12001->8080/tcp invoker0
2be1cec82dec whisk/controller:latest "/bin/sh -c 'expor..." About an hour ago Up About an hour 0.0.0.0:8000->2551/tcp, 0.0.0.0:10001->8080/tcp controller0
0b48b19bfac5 whisk/controller:latest "/bin/sh -c 'expor..." About an hour ago Up About an hour 0.0.0.0:8001->2551/tcp, 0.0.0.0:10002->8080/tcp controller1
3bbac39d2e79 ches/kafka:0.10.2.1 "/start.sh" About an hour ago Up About an hour 7203/tcp, 0.0.0.0:9092->9092/tcp kafka
e4cd9c1d4001 zookeeper:3.4 "/docker-entrypoin..." About an hour ago Up About an hour 2888/tcp, 0.0.0.0:2181->2181/tcp, 3888/tcp zookeeper
cc740909178b openwhisk/apigateway:0.8.2 "/usr/local/bin/du..." About an hour ago Up About an hour 80/tcp, 8423/tcp, 0.0.0.0:9000->9000/tcp, 0.0.0.0:9001->8080/tcp apigateway
c73a0ebf3305 redis:3.2 "docker-entrypoint..." About an hour ago Up About an hour 0.0.0.0:6379->6379/tcp redis
FAQ： $ go godep: error downloading dep (golang.org/x/sys/unix): unrecognized import path "golang.org/x/sys/unix" ===> https://github.com/Jimmy-Xu/incubator-openwhisk (patch)
Config CLI
- set apihost
- set auth key
$ cd ~/openwhisk
$ ./bin/wsk property set --apihost 172.17.0.1
ok: whisk API host set to 172.17.0.1
$ ./bin/wsk property set --auth `cat ansible/files/auth.guest`
ok: whisk auth set. Run 'wsk property get --auth' to see the new value.
use action
$ cd ~/openwhisk
$ ./bin/wsk -i sdk install docker
$ tree dockerSkeleton/
dockerSkeleton/
├── buildAndPush.sh
├── Dockerfile
├── example.c
└── README.md
$ cat dockerSkeleton/example.c
#include <stdio.h>
int main(int argc, char *argv[]) {
printf("This is an example log message from an arbitrary C program!\n");
printf("{ \"msg\": \"Hello from arbitrary C program!\", \"args\": %s }",
(argc == 1) ? "undefined" : argv[1]);
$ docker login -u xjimmyshcn -p <password>
$ cd dockerSkeleton
$ ./buildAndPush.sh xjimmyshcn/blackboxdemo
$ cd ..
$ docker images xjimmyshcn/blackboxdemo
REPOSITORY TAG IMAGE ID CREATED SIZE
xjimmyshcn/blackboxdemo latest 6b5a7df02bab 13 minutes ago 110MB
//creaete action
$ ./bin/wsk -i action create example --docker xjimmyshcn/blackboxdemo
ok: created action example
$ ./bin/wsk -i action list
actions
/guest/example private blackbox
//invoke action
$ ./bin/wsk -i action invoke --result example --param payload Hyper
or
$ ./bin/wsk -i action invoke --result /guest/example --param payload Hyper
{
"args": {
"payload": "Hyper"
},
"msg": "Hello from arbitrary C program!"
}
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
f3fa8e797890 xjimmyshcn/blackboxdemo "/bin/bash -c 'cd ..." 14 seconds ago Up 13 seconds wsk1_5_guest_example
//the container above will keep 10 minutes
$ docker logs wsk1_5_guest_example
This is an example log message from an arbitrary C program!
XXX_THE_END_OF_A_WHISK_ACTIVATION_XXX
XXX_THE_END_OF_A_WHISK_ACTIVATION_XXX
//watch dockerd log
$ sudo journalctl -u docker.service -f | grep Calling
Nov 17 06:18:34 osboxes dockerd[1863]: time="2017-11-17T06:18:34.990026565-05:00" level=debug msg="Calling POST /v1.24/images/create?fromImage=xjimmyshcn%2Fblackboxdemo&tag=latest"
Nov 17 06:18:40 osboxes dockerd[1863]: time="2017-11-17T06:18:40.130948535-05:00" level=debug msg="Calling POST /v1.24/containers/create?name=wsk1_5_guest_example"
Nov 17 06:18:40 osboxes dockerd[1863]: time="2017-11-17T06:18:40.352035305-05:00" level=debug msg="Calling POST /v1.24/containers/f3fa8e7978902237352f953eaccaae54007dc49e2c3a93030da40839d3af1e37/start"
Nov 17 06:28:45 osboxes dockerd[1863]: time="2017-11-17T06:28:45.128940066-05:00" level=debug msg="Calling DELETE /v1.24/containers/f3fa8e7978902237352f953eaccaae54007dc49e2c3a93030da40839d3af1e37?force=1"
use package
$ ./bin/wsk -i package list /guest
$ ./bin/wsk -i package list /whisk.system
packages
/whisk.system/weather shared
/whisk.system/watson-textToSpeech shared
/whisk.system/utils shared
/whisk.system/websocket shared
/whisk.system/watson-speechToText shared
/whisk.system/combinators shared
/whisk.system/samples shared
/whisk.system/watson-translator shared
/whisk.system/github shared
/whisk.system/slack shared
//create package： https://github.com/apache/incubator-openwhisk/blob/master/docs/packages.md#creating-a-package
$ ./bin/wsk -i package create custom
ok: created package custom
$ ./bin/wsk -i package get --summary custom
package /guest/custom
(parameters: none defined)
//Share a package
$ ./bin/wsk -i package update custom --shared yes
ok: updated package custom
$ ./bin/wsk -i package get custom | grep publish
"publish": true,
//create action in custom package
$ ./bin/wsk -i action create custom/identity identity.js
ok: created action custom/identity
$ ./bin/wsk -i action create custom/example --docker xjimmyshcn/blackboxdemo
$ ./bin/wsk -i action invoke --result custom/example --param payload Hyper
invoke action via curl
export NAMESPACE=guest
export AUTH='Authorization: Basic '`./bin/wsk -i property get --auth | awk '{printf("%s", $3)}' | base64 --wrap=0`
export TYPE="Content-Type: application/json"
$ curl -s -k -H "$TYPE" -H "$AUTH" https://172.17.0.1/api/v1/namespaces/$NAMESPACE | jq ".packages"
[
{
"name": "custom",
"binding": false,
"publish": true,
"annotations": [],
"version": "0.0.3",
"namespace": "guest"
}
]
$ curl -s -k -H "$TYPE" -H "$AUTH" https://172.17.0.1/api/v1/namespaces/$NAMESPACE/actions | jq "."
[
{
"name": "example",
"publish": false,
"annotations": [
{
"key": "exec",
"value": "blackbox"
}
],
"version": "0.0.1",
"namespace": "guest/custom"
},
{
"name": "identity",
"publish": false,
"annotations": [
{
"key": "exec",
"value": "nodejs:6"
}
],
"version": "0.0.1",
"namespace": "guest/custom"
},
{
"name": "example",
"publish": false,
"annotations": [
{
"key": "exec",
"value": "blackbox"
}
],
"version": "0.0.1",
"namespace": "guest"
},
{
"name": "hello",
"publish": false,
"annotations": [
{
"key": "exec",
"value": "nodejs:6"
}
],
"version": "0.0.1",
"namespace": "guest"
}
]
//asynchronous
$ curl -s -k -H "$TYPE" -H "$AUTH" -X POST -d '{"payload":"Hyper"}' https://172.17.0.1/api/v1/namespaces/$NAMESPACE/actions/example
{"activationId":"4936a64ad1c14f2db6a64ad1c1ff2d1d"}
//synchronous
$ curl -s -k -H "$TYPE" -H "$AUTH" -X POST -d '{"payload":"Hyper"}' https://172.17.0.1/api/v1/namespaces/$NAMESPACE/actions/example?blocking=true | jq ".response.result"
{
"args": {
"payload": "Hyper"
},
"msg": "Hello from arbitrary C program!"
}