Skip to content

Instantly share code, notes, and snippets.

@manasdk

manasdk/1-multi_node_deploy.md Secret

Last active March 9, 2016 02:48
Show Gist options
  • Save manasdk/fce14029900e533a385d to your computer and use it in GitHub Desktop.
Save manasdk/fce14029900e533a385d to your computer and use it in GitHub Desktop.
Download ZIP
st2 multi node take 2
Raw
1-multi_node_deploy.md

Deploy 3 boxes using ubuntu14 (st2-multi-node-controller, st2-multi-node-1 and st2-multi-node-2)

st2-multi-node-controller

  • mongodb
  • rabbitmq
  • postgres
  • load balancing nginx

st2-multi-node-1

  • st2

st2-multi-node-2

  • st2

Steps

st2-multi-node-controller

install components

sudo apt-get install -y mongodb-server rabbitmq-server postgresql

Configure postgresql for remote connection

  1. Fix listen address in /etc/postgresql/9.3/main/postgresql.conf

  2. To /etc/postgresql/9.3/main/pg_hba.conf add -

host       all  all  10.0.3.1/24  trust

  1. restart postgresql

  2. Create Mistral DB in postgresql (per https://docs.stackstorm.com/install/deb.html#setup-mistral-database) -

 cat << EHD | sudo -u postgres psql
 CREATE ROLE mistral WITH CREATEDB LOGIN ENCRYPTED PASSWORD 'StackStorm';
 CREATE DATABASE mistral OWNER mistral;
 EHD
  1. Setup st2web and SSL termination. Follow https://docs.stackstorm.com/install/deb.html#install-webui-and-setup-ssl-termination. Will need to 
    curl -s https://packagecloud.io/install/repositories/StackStorm/staging-stable/script.deb.sh | sudo bash
  2. nginx config for this box from https://gist.github.com/manasdk/fce14029900e533a385d#file-shared_st2_nginx-conf
  3. Install st2chatops following from https://docs.stackstorm.com/latest/install/deb.html#setup-chatops

mongodb

Change bind_ip in /etc/monogodb.conf and service mongodb restart

rabbitmq

No change required

st2-multi-node-1

  1. curl -s https://packagecloud.io/install/repositories/StackStorm/staging-stable/script.deb.sh | sudo bash
  2. sudo apt-get install -y st2 st2mistral
  3. Fix database.connection in /etc/mistral/mistral.conf
  4. Fix default.transport_url in /etc/mistral/mistral.conf
  5. Setup user as per https://docs.stackstorm.com/install/deb.html#configure-ssh-and-sudo
  6. Use shared st2 config https://gist.github.com/manasdk/fce14029900e533a385d#file-st2-conf
  7. Configure authentication https://docs.stackstorm.com/install/deb.html#configure-authentication
  8. nginx config for this box from https://gist.github.com/manasdk/fce14029900e533a385d#file-st2_nginx-conf

st2-multi-node-2

  1. curl -s https://packagecloud.io/install/repositories/StackStorm/staging-stable/script.deb.sh | sudo bash
  2. sudo apt-get install -y st2 st2mistral
  3. Fix database.connection in /etc/mistral/mistral.conf
  4. Setup user as per https://docs.stackstorm.com/install/deb.html#configure-ssh-and-sudo
  5. Use shared st2 config https://gist.github.com/manasdk/fce14029900e533a385d#file-st2-conf
  6. Configure authentication https://docs.stackstorm.com/install/deb.html#configure-authentication
  7. nginx config for this box from https://gist.github.com/manasdk/fce14029900e533a385d#file-st2_nginx-conf
Raw
shared_st2_nginx.conf
#
# nginx configuration to expose st2 webui, redirect HTTP->HTTPS,
# provide SSL termination, and reverse-proxy st2api and st2auth API endpoint.
# To enable:
# cp ${LOCATION}/st2.conf /etc/nginx/sites-available
# ln -l /etc/nginx/sites-available/st2.conf /etc/nginx/sites-enabled/st2.conf
# see https://docs.stackstorm.com/install.html for details
upstream st2 {
server st2-multi-node-1:443;
server st2-multi-node-2:443;
}
server {
listen *:80 default_server;
add_header Front-End-Https on;
add_header X-Content-Type-Options nosniff;
if ($ssl_protocol = "") {
return 301 https://$host$request_uri;
}
index index.html index.htm index.php;
access_log /var/log/nginx/st2webui.access.log combined;
error_log /var/log/nginx/st2webui.error.log;
}
server {
listen *:443 ssl;
ssl on;
ssl_certificate /etc/ssl/st2/st2.crt;
ssl_certificate_key /etc/ssl/st2/st2.key;
ssl_session_cache shared:SSL:10m;
ssl_session_timeout 5m;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH:ECDHE-RSA-AES128-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA128:DHE-RSA-AES128-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES128-GCM-SHA128:ECDHE-RSA-AES128-SHA384:ECDHE-RSA-AES128-SHA128:ECDHE-RSA-AES128-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES128-SHA128:DHE-RSA-AES128-SHA128:DHE-RSA-AES128-SHA:DHE-RSA-AES128-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES128-GCM-SHA384:AES128-GCM-SHA128:AES128-SHA128:AES128-SHA128:AES128-SHA:AES128-SHA:DES-CBC3-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!MD5:!PSK:!RC4;
ssl_prefer_server_ciphers on;
index index.html index.htm index.php;
access_log /var/log/nginx/ssl-st2webui.access.log combined;
error_log /var/log/nginx/ssl-st2webui.error.log;
add_header Front-End-Https on;
add_header X-Content-Type-Options nosniff;
location /api/ {
rewrite ^/api/(.*) /api/$1 break;
proxy_pass https://st2/api/;
proxy_read_timeout 90;
proxy_connect_timeout 90;
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Connection '';
chunked_transfer_encoding off;
proxy_buffering off;
proxy_cache off;
proxy_set_header Host $host;
}
location /auth/ {
rewrite ^/auth/(.*) /auth/$1 break;
proxy_pass https://st2/auth/;
proxy_read_timeout 90;
proxy_connect_timeout 90;
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass_header Authorization;
proxy_set_header Connection '';
chunked_transfer_encoding off;
proxy_buffering off;
proxy_cache off;
proxy_set_header Host $host;
}
location /mistral/ {
rewrite ^/mistral/(.*) /mistral/$1 break;
proxy_pass https://st2/mistral/;
proxy_read_timeout 90;
proxy_connect_timeout 90;
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass_header Authorization;
proxy_set_header Connection '';
chunked_transfer_encoding off;
proxy_buffering off;
proxy_cache off;
proxy_set_header Host $host;
}
location / {
root /opt/stackstorm/static/webui/;
index index.html index.htm index.php;
}
}
Raw
st2.conf
# System-wide configuration
[api]
# Host and port to bind the API server.
host = 127.0.0.1
port = 9101
logging = /etc/st2/logging.api.conf
mask_secrets = True
# allow_origin is required for handling CORS in st2 web UI.
# allow_origin = http://myhost1.example.com:3000,http://myhost2.example.com:3000
[sensorcontainer]
logging = /etc/st2/logging.sensorcontainer.conf
[rulesengine]
logging = /etc/st2/logging.rulesengine.conf
[actionrunner]
logging = /etc/st2/logging.actionrunner.conf
virtualenv_opts = --always-copy
[resultstracker]
logging = /etc/st2/logging.resultstracker.conf
[notifier]
logging = /etc/st2/logging.notifier.conf
[exporter]
logging = /etc/st2/logging.exporter.conf
[garbagecollector]
logging = /etc/st2/logging.garbagecollector.conf
[auth]
host = 127.0.0.1
port = 9100
use_ssl = False
debug = False
enable = True
logging = /etc/st2/logging.auth.conf
mode = standalone
# Note: Settings bellow are only used in "standalone" mode
backend = flat_file
backend_kwargs = {"file_path": "/etc/st2/htpasswd"}
# Base URL to the API endpoint excluding the version (e.g. http://myhost.net:9101/)
api_url =
[system]
base_path = /opt/stackstorm
[syslog]
host = st2-multi-node-controller
port = 514
facility = local7
protocol = udp
[log]
excludes = requests,paramiko
redirect_stderr = False
mask_secrets = True
[system_user]
user = stanley
ssh_key_file = /home/stanley/.ssh/stanley_rsa
[messaging]
url = amqp://guest:guest@st2-multi-node-controller:5672/
[ssh_runner]
remote_dir = /tmp
use_paramiko_ssh_runner = True
[database]
host = st2-multi-node-controller
[mistral]
v2_base_url = https://st2-multi-node-controller/mistral/v2
api_url = https://st2-multi-node-controller/api
Raw
st2_nginx.conf
#
# nginx configuration to expose st2 webui, redirect HTTP->HTTPS,
# provide SSL termination, and reverse-proxy st2api and st2auth API endpoint.
# To enable:
# cp ${LOCATION}/st2.conf /etc/nginx/sites-available
# ln -l /etc/nginx/sites-available/st2.conf /etc/nginx/sites-enabled/st2.conf
# see https://docs.stackstorm.com/install.html for details
server {
listen *:80 default_server;
add_header Front-End-Https on;
add_header X-Content-Type-Options nosniff;
if ($ssl_protocol = "") {
return 301 https://$host$request_uri;
}
index index.html index.htm index.php;
access_log /var/log/nginx/st2webui.access.log combined;
error_log /var/log/nginx/st2webui.error.log;
}
server {
listen *:443 ssl;
ssl on;
ssl_certificate /etc/ssl/st2/st2.crt;
ssl_certificate_key /etc/ssl/st2/st2.key;
ssl_session_cache shared:SSL:10m;
ssl_session_timeout 5m;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH:ECDHE-RSA-AES128-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA128:DHE-RSA-AES128-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES128-GCM-SHA128:ECDHE-RSA-AES128-SHA384:ECDHE-RSA-AES128-SHA128:ECDHE-RSA-AES128-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES128-SHA128:DHE-RSA-AES128-SHA128:DHE-RSA-AES128-SHA:DHE-RSA-AES128-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES128-GCM-SHA384:AES128-GCM-SHA128:AES128-SHA128:AES128-SHA128:AES128-SHA:AES128-SHA:DES-CBC3-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!MD5:!PSK:!RC4;
ssl_prefer_server_ciphers on;
index index.html index.htm index.php;
access_log /var/log/nginx/ssl-st2webui.access.log combined;
error_log /var/log/nginx/ssl-st2webui.error.log;
add_header Front-End-Https on;
add_header X-Content-Type-Options nosniff;
location /api/ {
rewrite ^/api/(.*) /$1 break;
proxy_pass http://127.0.0.1:9101/;
proxy_read_timeout 90;
proxy_connect_timeout 90;
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Connection '';
chunked_transfer_encoding off;
proxy_buffering off;
proxy_cache off;
proxy_set_header Host $host;
}
location /stream/ {
rewrite ^/stream/(.*) /$1 break;
proxy_pass http://127.0.0.1:9102/;
proxy_read_timeout 90;
proxy_connect_timeout 90;
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass_header Authorization;
sendfile on;
tcp_nopush on;
tcp_nodelay on;
# Disable buffering and chunked encoding.
# In the stream case we want to receive the whole payload at once, we don't
# want multiple chunks.
proxy_set_header Connection '';
chunked_transfer_encoding off;
proxy_buffering off;
proxy_cache off;
proxy_set_header Host $host;
}
# For backward compatibility reasons, rewrite requests from "/api/stream"
# to "/stream/v1/stream" and "/api/v1/stream" to "/stream/v1/stream"
location /api/stream/ {
rewrite ^/api/stream/?(.*)$ /v1/stream/$1 break;
proxy_pass http://127.0.0.1:9102;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
sendfile on;
tcp_nopush on;
tcp_nodelay on;
# Disable buffering and chunked encoding.
# In the stream case we want to receive the whole payload at once, we don't
# want multiple chunks.
proxy_set_header Connection '';
chunked_transfer_encoding off;
proxy_buffering off;
proxy_cache off;
}
location /api/v1/stream/ {
rewrite ^/api/v1/stream/?(.*)$ /v1/stream/$1 break;
proxy_pass http://127.0.0.1:9102;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
sendfile on;
tcp_nopush on;
tcp_nodelay on;
# Disable buffering and chunked encoding.
# In the stream case we want to receive the whole payload at once, we don't
# want multiple chunks.
proxy_set_header Connection '';
chunked_transfer_encoding off;
proxy_buffering off;
proxy_cache off;
}
location /auth/ {
rewrite ^/auth/(.*) /$1 break;
proxy_pass http://127.0.0.1:9100/;
proxy_read_timeout 90;
proxy_connect_timeout 90;
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass_header Authorization;
proxy_set_header Connection '';
chunked_transfer_encoding off;
proxy_buffering off;
proxy_cache off;
proxy_set_header Host $host;
}
location /mistral/ {
rewrite ^/mistral/(.*) /$1 break;
proxy_pass http://127.0.0.1:8989/;
proxy_read_timeout 90;
proxy_connect_timeout 90;
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass_header Authorization;
proxy_set_header Connection '';
chunked_transfer_encoding off;
proxy_buffering off;
proxy_cache off;
proxy_set_header Host $host;
}
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment