Skip to content

Instantly share code, notes, and snippets.

@jsteenb2
Last active March 24, 2020 23:36
Show Gist options
  • Save jsteenb2/6691d6b3e0bef5c5168774c2c8c6ad67 to your computer and use it in GitHub Desktop.
Save jsteenb2/6691d6b3e0bef5c5168774c2c8c6ad67 to your computer and use it in GitHub Desktop.
Description of Pkger Stacks

Pkger: the What and How

Responsibilities

  • Translating a declarative package file (either JSON | Yaml | Jsonnet) into resources in the platform
  • Exporting existing resources in the form of a pkg (either JSON | Yaml)
  • Managing the state of a pkg's side effects via a stack

Anatomy of a package

A package is a collection of resource configurations. These resource configurations can be seen in full in the pkger/testdata directory. The package itself does have any state. Packages may contain resources that are not uniquely identifiable within the platform and some that are not. If it is desired to use packages in a gitops scenario or in a manner that requires all resources are not duplicated, you will want to explore using a stack.

Stacks

A stack is a stateful entity for which packages can be applied and managed. A stack uses a combination of a resource's kind and metadata.name fields to uniquely identify a resource inside a package and map that to state in the platform. Via this state, a stack provides the ability to apply a package idempotently. Stack's manage the full lifecyle of a package's resources, including creating, updating, and deleting resources.

Packages may contain resources that are not uniquely identifiable within the platform. For instance, a dashboard resource, does not have any unique identifier within the platform beyond its UUID. A stack uses the metadata.name field to uniquely identify a resource inside a package and map that to state in the platform.

Stacks will manage the following use cases:

We create a stack without any URLs to packages, henceforth identified as S1:

# S1 package - initial
kind: Label
metadata:
	name: lucid_einstein
spec:
	name: label_1
---
kind: Bucket
metadta: 
	name: pristine_noir
spec: 
	name: bucket_1
	association:
	 	- kind: Label
		   name: lucid_einstein
---
kind: Dashboard
metadata:
	name: charmed_saratoba
spec:
	name: dash_1
	association:
	 	- kind: Label
		   name: lucid_einstein
  1. The S1 package (seen above) with all new resources is applied

    • Side effects: all resources are created and a record of all resources (id, res type, etc) is added to the S1 stack record

      Stack Record
         {
           "stack_id": S1_UUID,
           "createdAt": CreatedAtTimestamp,
           "updatedAt": CreatedAtTimestamp,
           "config": {},
           "resources": [
             {
               "kind": "Label",
               "id": LABEL_UUID,
               "pkgName": "lucid_einstein"
             },
             {
               "kind": "Bucket",
               "id": BUCKET_UUID,
               "pkgName": "pristine_noir",
               "associations": [
                 {
                   "kind": "Label",
                   "pkgName": "lucid_einstein"
                 }
               ]
             },
             {
               "kind": "Dashboard",
               "id": DASHBOARD_UUID,
               "pkgName": "charmed_saratoba",
               "associations": [
                 {
                   "kind": "Label",
                   "pkgName": "lucid_einstein"
                 }
               ]
             }
           ]
         }  
  2. Same S1 package (seen above) is reapplied with no changes from step 1

    • Side effects: nothing, no changes

      Stack Record
        {
          "stack_id": S1_UUID,
          "createdAt": CreatedAtTimestamp,
          "updatedAt": CreatedAtTimestamp,
          "config": {},
          "resources": [
            {
              "kind": "Label",
              "id": LABEL_UUID,
              "pkgName": "lucid_einstein"
            },
            {
              "kind": "Bucket",
              "id": BUCKET_UUID,
              "pkgName": "pristine_noir",
              "associations": [
                {
                  "kind": "Label",
                  "pkgName": "lucid_einstein"
                }
              ]
            },
            {
              "kind": "Dashboard",
              "id": DASHBOARD_UUID,
              "pkgName": "charmed_saratoba",
              "associations": [
                {
                  "kind": "Label",
                  "pkgName": "lucid_einstein"
                }
              ]
            }
          ]
        }  


# S1 package - updated label name
kind: Label
metadata:
	name: lucid_einstein
spec:
	name: cool label name  #<<<<<< THIS NAME CHANGES
---
kind: Bucket
metadta: 
	name: pristine_noir
# snip - no changes
---
kind: Dashboard
metadata:
	name: charmed_saratoba
# snip - no changes
  1. The S1 package is applied with an update to the label resource

    • Side effects: platform label (LABEL_UUID) is renamed and updatedAt field in S1 record is updated

      Stack Record
        {
             "stack_id": S1_UUID,
             "createdAt": CreatedAtTimestamp,
             "updatedAt": LABEL_UPDATE_TIMESTAMP,
             "config": {},
             "resources": [
               ... snip, all resoruces are same
             ]
        }  


# S1 package - new reosource added
kind: Label
metadata:
	name: lucid_einstein
# snip - no change
---
kind: Bucket
metadta: 
	name: pristine_noir
# snip - no changes
---
kind: Dashboard
metadata:
	name: charmed_saratoba
# snip - no changes
---
kind: Task  #<<<<<< THIS RESOURCE IS ADDED
metadata:
	name: alcord_mumphries
spec:
	name: task_1
	association:
	 	- kind: Label
		   name: lucid_einstein
  1. The S1 package is applied with a new resource added

    • Side effects: new task is created and S1 record is updated

      Stack Record
        {
             "stack_id": S1_UUID,
             "createdAt": CreatedAtTimestamp,
             "updatedAt": TASK_ADD_TIMESTAMP,
             "config": {},
             "resources": [
               ... snip, all resoruces from before,
               {
               	"kind": "Task",
               	"id": TASK_UUID,
         				"pkgName": "alcord_mumphries",
                 "associations": [
                   {
                     "kind": "Label",
                     "pkgName": "lucid_einstein"
                   }
                 ]
               }
             ]
        }  


# S1 package - task resource is removed
kind: Label
metadata:
	name: lucid_einstein
# snip - no change
---
kind: Bucket
metadta: 
	name: pristine_noir
# snip - no changes
---
kind: Dashboard
metadata:
	name: charmed_saratoba
# snip - no changes
  1. The S1 package is applied with changes that removes an existing resource
    • Side effects: task is deleted from platform and S1 record is updated

      Stack Record
        {
            "stack_id": S1_UUID,
            "createdAt": CreatedAtTimestamp,
            "updatedAt": TASK_DELETE_TIMESTAMP,
            "config": {},
            "resources": [
              {
               "kind": "Label",
               "id": LABEL_UUID,
               "pkgName": "lucid_einstein"
             },
             {
               "kind": "Bucket",
               "id": BUCKET_UUID,
               "pkgName": "pristine_noir",
               "associations": [
                 {
                   "kind": "Label",
                   "pkgName": "lucid_einstein"
                 }
               ]
             },
             {
               "kind": "Dashboard",
               "id": DASHBOARD_UUID,
               "pkgName": "charmed_saratoba",
               "associations": [
                 {
                   "kind": "Label",
                   "pkgName": "lucid_einstein"
                 }
               ]
             }
            ]
        }  


# S1 package - label and associations to it are removed
kind: Bucket
metadta: 
	name: pristine_noir
spec: 
	name: bucket_1
---
kind: Dashboard
metadata:
	name: charmed_saratoba
spec:
	name: dash_1
  1. The S1 package is apllied with label and associations to that label removed
    • Side effects: label and all label assocations for that label are removed from the platform and S1 record is updated

      Stack Record
       {
           "stack_id": S1_UUID,
           "createdAt": CreatedAtTimestamp,
           "updatedAt": Label_DELETE_TIMESTAMP,
           "config": {},
           "resources": [
            {
              "kind": "Bucket",
              "id": BUCKET_UUID,
              "pkgName": "pristine_noir",
              "associations": []
            },
            {
              "kind": "Dashboard",
              "id": DASHBOARD_UUID,
              "pkgName": "charmed_saratoba",
              "associations": []
            }
           ]
       }  
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment