Skip to content

Instantly share code, notes, and snippets.

@xofer
Last active June 28, 2021 11:43
Show Gist options
  • Save xofer/e2a703d80979108c76ce53e5361fbc4d to your computer and use it in GitHub Desktop.
Save xofer/e2a703d80979108c76ce53e5361fbc4d to your computer and use it in GitHub Desktop.
Storm UI Nginx conf for Nimbus with logviewer proxy

We have a setup that I assume is quite common: A publicly accessible Nimbus running Storm UI. The worker nodes can only be accessed from the Nimbus (via the LAN). All the nodes have internal DNS names (i.e. node.lan.example.com), which is set in the configuration files; they use these DNS names to reach each other. The Nimbus has an external DNS name (storm.example.com) for public access. The Nimbus's UI is behind an Nginx proxy, which provides HTTP Auth and HTTPS.

Because of this setup, the logviewer links in the UI do not work. In order to fix this, we employ an elaborate hack shown in the conf file below. It uses ngx_http_substitutions_filter_module to rewrite content returned by the Storm UI and some complicated URL rewrite tricks to proxy the workers' logviewers through through the Nimbus.

A particularly difficult problem to solve was introduced by the urlencoded slashes in file paths. Getting that to pass through the proxy with the rest of the URL intact was not easy. The solution is the final block in the file.

server {
# snip
# standard stuff: listen, https, logging, auth, etc.
# /snip
location / {
subs_filter_types application/json;
subs_filter http:\\\/\\\/[^.]*-head\.lan\.example\.com:8000 https:\/\/$host gr;
subs_filter http:\\\/\\\/([^.]*)\.lan\.example\.com:8000 https:\/\/$host\/proxy\/$1 gr;
subs_filter "http://{{host}}:{{logviewerPort}}" "https://$host/proxy/{{host}}" g;
proxy_set_header Accept-Encoding "";
proxy_pass http://127.0.0.1:8081/;
}
location = /log {
proxy_pass http://127.0.0.1:8000/daemonlog;
}
location /daemonlog {
proxy_pass http://127.0.0.1:8000/daemonlog;
}
location /daemondownload {
proxy_pass http://127.0.0.1:8000/daemondownload;
}
location /search {
subs_filter_types application/json;
subs_filter http:\/\/localhost:8000 https:\/\/$host g;
subs_filter 'http://"+host+":"+logviewerPort' 'https://$host/proxy/"+host' g;
proxy_set_header Accept-Encoding "";
proxy_pass http://127.0.0.1:8000/search;
}
location ~ /proxy/(?<proxyhost>[^/.]*)[^/]*/(?<rest>.*) {
subs_filter_types application/json;
subs_filter '"/log?' '"/proxy/$proxyhost/daemonlog?' g;
subs_filter '"/daemondownload/' '"/proxy/$proxyhost/daemondownload/' g;
subs_filter '"/logviewer_search.html?' '"/proxy/$proxyhost/logviewer_search.html?' g;
subs_filter '"/search/' '"/proxy/$proxyhost/search/' g;
subs_filter http:\/\/localhost:8000 https:\/\/$host/proxy/$proxyhost g;
proxy_set_header Accept-Encoding "";
resolver 127.0.0.1; # required for proxy_pass with variable
# we need to pass urlencoded values in $uri and $args, without encoding the
# question mark that delimits the query params portion
rewrite ^ $request_uri;
rewrite ^/proxy/[^/.]*[^/]*/(.*)$ $1? break;
proxy_pass http://$proxyhost.lan.example.com:8000/$uri$is_args$args;
}
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment