Skip to content

Instantly share code, notes, and snippets.

@frimik
Last active August 23, 2017 14:56
Show Gist options
  • Save frimik/32081ea966ad5fc016257058807a52f7 to your computer and use it in GitHub Desktop.
Save frimik/32081ea966ad5fc016257058807a52f7 to your computer and use it in GitHub Desktop.
Aurora Scheduler / Mesos Master SSL and Auth solution

About

Aurora Scheduler consists of 5 instances across 5 nodes usually. Usually there is a zookeeper instance on each of those 5. Maybe you also have 5x Mesos Master instances on the same nodes.

Now you want SSL ... and AUTH on those!? Tough luck ...

The documentation discuss making the scheduler listen on 127.0.0.1:8081 and putting an nginx on each of the nodes listening on 0.0.0.0:8081 or the node IP. I forget which. Now you'll think - ok - what happens to "sane hostnames", "hostnames matching certificate CN"?

Look no further

The concept is simple, the implementations can vary in which components one uses. I will list an example one:

First iteration

  • Standard Nginx - point a new DNS record to its IP address: aurora.example.com
  • Make it configured for HTTPS, with whatever fancy certificate you have.
  • Enabled Basic auth and create a nice aurora.htpasswd file.
  • Configure it to proxy to a backend DNS record: leader-aurora.my.net
  • Create a nice little python? script which fetches the http://your-aurora-nodes:8081/scheduler address and extract the redirect target host name. Have the script update DNS if the new leader doesn't match the currently DNS-set leader leader-aurora.example.com.
  • Now you'll have the currently elected leader always available on the https://aurora.example.com/ address.

Second iteration

  • Replace the standard Nginx with https://github.com/rstudio/redx
  • Fancy up your python script to instead be a zookeeper-watcher (zk:/aurora/scheduler) which updates this Redis back-end "in real time" via its REST API!

Third iteration

  • Plug the potential race condition where a leader election may happen while a request is coming in to the aurora scheduler by triggering the leader-lookup explicitly on a 307 redirect (Aurora non-elected scheduler sends those on http requests).

Aurora CLI configuration:

  • ~/.aurora/clusters.json:
    scheduler_uri: https://aurora.example.com
    
  • ~/.netrc:
    machine aurora.example.com
    login username
    password password
    
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment