Skip to content

Instantly share code, notes, and snippets.

@CrashenX
Last active August 29, 2015 14:08
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save CrashenX/8fc6d42ffc154ae0682b to your computer and use it in GitHub Desktop.
Save CrashenX/8fc6d42ffc154ae0682b to your computer and use it in GitHub Desktop.
OpenStack Image Objects Prescribed Architecture

Architecture

High-Level Architecture

 +------------+
 |            |
 |    Nova  +----------------------+
 |          | |                    |
 +------------+ Glance Middleware  +----+
            |                      |    |
            +----------------------+    |
                                        |
                                        |
                                        |
 +------------+                         |
 |            |                         |
 |    Nova  +----------------------+    +----------+    +----------------+
 |          | |                    |    |          |    |                |
 +------------+ Glance Middleware  +----+  Router  +----+  Object Store  |
            |                      |    |          |    |                |
            +----------------------+    +----------+    +----------------+
                                        |
 ...                                    |
                                        |
                                        |
 +------------+                         |
 |            |                         |
 |    Nova  +----------------------+    |
 |          | |                    |    |
 +------------+ Glance Middleware  +----+
            |                      |
            +----------------------+

Objects

Immutability

Objects are immutable, and every object has a UUID. In order to modify (i.e. mutate) an object, a new object has to be created with the desired mutations and a new UUID.

Optimization: Copying large amounts of data is inefficient, so techniques such as filesystem-level hardlinking should be employed. Note that this is safe because the objects are immutable.

Types

Types define the structure of the data. There are two types: OS (Operating System) and User.

OS

{
    etc/
        .git # Transaction Log
        configs/
            OS Config
            App 1 Config
            App 2 Config
            ...
            App N Config
        metadata/
            UUID
            ...
            Representation of Current Disk Hierarchy
    data/
        disks/
            Diff Disk 1 # No Parent (Base Image)
            Diff Disk 2 # Contains Reference to Parent
            Diff Disk 3 # Contains Reference to Parent
            Diff Disk 4 # Contains Reference to Parent
            ...
            Diff Disk N # Contains Reference to Parent
            Latest Disk # No Parent (Coalesced Chain)
}

User

{
    etc/
        .git # Transaction Log
        configs/
            OS Config
            App 1 Config
            App 2 Config
            ...
            App N Config
        metadata/
            UUID
            Current OS Disk UUID
            ...
            Representation of Current Overlay Hierarchy
    data/
        overlays/
            Restore Point Overlay 1 # No Parent
            Restore Point Overlay 2 # Contains Reference to Parent (e.g. 1)
            Restore Point Overlay 3 # Contains Reference to Parent (e.g. 1)
            Restore Point Overlay 4 # Contains Reference to Parent (e.g. 2)
            …
            Restore Point Overlay N # Contains Reference to Parent (e.g. 1)
            Active Overlay # Contains Reference to Parent (e.g. 3)
            Ephemeral Overlay # Contains Reference to Active Overlay
}

Glance Middleware

Operations

  • filter: obtain a subset of objects based on the specified criteria
  • list: obtain the set of all objects on the specified store
  • download: create a local copy of an object from the remote store
  • upload: copy a local object onto the remote store
  • delete: delete an object from the remote store

Optimization: Filtering locally will result in downloading an unecessary amount of data. Filters should be applied on the remote object store.

Nova

Operations

VM Up

  1. If ephemeral overlay exists, emit notification and enter recovery mode
  2. Get latest OS disk
  3. Boot it
  4. Apply OS configs
  5. Install OS apps
  6. Configure OS apps
  7. Apply user specific OS configs
  8. Install user apps
  9. Configure user apps
  10. Apply any restore point overlay ancestors of active overlay to disk
  11. Apply active overlay to disk
  12. Create ephemeral overlay
  13. Union mount ephemeral overlay
  14. Run tests
    • If pass, update current OS disk ID and commit transaction
    • If fail, emit notification
      • If loaded disk is not current disk, delete ephemeral overlay, load current disk, and goto 2

VM Down

  • Write ephemeral overlay to active overlay
  • Delete ephemeral overlay

Create Restore Point

  • Convert active overlay to restore point overlay
  • Create new active overlay
  • Update representation of current overlay hierarchy in etc/metadata/
  • Commit transaction

Delete Restore Point

  • Merge restore point overlay into child, if exists; else delete
  • Update representation of current overlay hierarchy in etc/metadata/
  • Commit transaction
@jaypipes
Copy link

Does "OS" above refer to OpenStack or Operating System?

@CrashenX
Copy link
Author

Operating System. I added clarification.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment