Skip to content

Instantly share code, notes, and snippets.

@Funcan
Created February 10, 2016 17:26
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save Funcan/321e955ee4990f3c3ed2 to your computer and use it in GitHub Desktop.
Save Funcan/321e955ee4990f3c3ed2 to your computer and use it in GitHub Desktop.
Idea for job distribution for H/A in cinder

The Point

Allow cinder to move to an A/A c-vol model, with minimal disruption and with a safe upgrade path

The Idea

  1. Add a 'cluster' attribute to every c-vol service.

    This is set via cinder.conf and must be the same on every node that services a specific backend. The deployer is required to get them right

  2. Teach the scheduler to keep track of clusters, and cluster to host mappings.

    At this point, creates can go to the cluster from an RPC point of view, but the scheduler always fills in a specific host name in the volume table 'host' field as well as the cluster name. The host can be a random one from the cluster, always the first one up, whatever. The create RPC does not need to end up on that node

At this point, create is A/A, but nothing else is

  1. Start moving the API calls to always go via the scheduler.

    As part of this process, we can get the scheduler to re-write the call to a specific host to go to the cluster instead. Each call (attach, detach, snap, etc) becomes H/A compatible one by one

I think this makes upgrade easier. Old API nodes will still have a host field they can use, old systems that overwrite the host field on multiple nodes behave the same as before, and we can move API calls one at a time, solving races as such as we find them.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment