Skip to content

Instantly share code, notes, and snippets.

@StanAngeloff
Last active August 29, 2015 14:10
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save StanAngeloff/150360f5b9d8027a1aa6 to your computer and use it in GitHub Desktop.
Save StanAngeloff/150360f5b9d8027a1aa6 to your computer and use it in GitHub Desktop.

Redundancy and Fail-over

MySQL

Master-Master in Active-Passive Mode

Two MySQL servers, each replicating data to the other. Any INSERT, UPDATE, etc. operations are transmitted over the network to the other instance. The two servers are local to each node and only the primary node gets all the HTTP/s connections. This is done so we can ensure only one web server accepts connections and writes to its local MySQL database.

Any change gets written to active server's binary log and flows through replication to the passive server's relay log. The passive server executes the query and writes the event to its own binary log, because the option log_slave_updates is enabled. The active server then retrieves the same change via replication into its own relay log, but ignores it because the server ID in the event matches its own.

Fail-over

When the primary server fails (all services are assumed to be down), the node balancer will take it out of rotation and start sending any HTTP/s requests to the secondary node. The secondary node will then start executing any requests and perform writes to its local database. When the primary server comes back online, the node balancer will put it back in rotation and the MySQL daemon will need to pick up any updates which have occurred whilst the node was done from the secondary master. The two servers then switch roles, the primary becomes the hot-spare and the secondary is now primary.

File system

For shared files, such as images, documents, etc. we need to have a common place to store them, e.g., GlusterFS.

Sessions

No sessions stored in files, a Redis node can take care of sharing sessions across the primary and secondary node.

Web server

In order for Nginx to be able to terminate SSL traffic, we need to proxy each request to an HTTP-only instance. E.g., we listen on port 80 for the default server which requires SSL and redirects to the www. domain. The 443 server then uses an upstream to proxy the request in HTTP mode to a server on 8080.

The reasoning behind this is to allow additional node balancers to communicate with the web server without enforcing SSL. E.g., if we chose to use HAProxy it will require all traffic from the node balancer to the web server to be over a private network IP and unencrypted, i.e., HTTP.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment