Skip to content

Instantly share code, notes, and snippets.

@ryanseys

ryanseys/compute-design-doc.md

Last active August 29, 2015 14:13
Show Gist options
  • Save ryanseys/28c9a17fea00899f3ac7 to your computer and use it in GitHub Desktop.
Save ryanseys/28c9a17fea00899f3ac7 to your computer and use it in GitHub Desktop.
Download ZIP
Google Compute Engine + gcloud-node Design Doc
Raw
compute-design-doc.md

Compute Engine + gcloud-node Design Doc

var gcloud = require('gcloud');

Heirarchy

Global level

There's the global level which handles all global objects available across the entire project such as images, networks, firewalls etc.

Zone level

Spawning off the global level (strangely, not off the region level) is the zone level, this contains resources that may be different from zone to zone, but are guaranteed are to be accessible within the zone. These are things like instances, disks and machine types.

I believe it makes the most sense to have a level for this level of abstraction so compute.zone('my-zone') or compute.defaultZone will go before any zone-level operation, ensuring that a zone is set or known ahead of time. This also teaches the developer the abstraction at the same time, which may be difficult to understand from the flat structure of the API.

Region level

Spawning off the global level is the region level, which contains resources that may be different from zone to zone but are guaranteed to be the same within the region. This includes things like regional operations, addresses and vpn tunnels.

Like the zone-level, abstracting the region-level operations to the own level of abstraction will help developers to understand the heirarchy and provide a level of sanity to the debugging process. For this reason, compute.region('my-region') or compute.defaultRegion will go before any region-level operation.

Operations

There are Operations at the global, zone and region levels. These are long-running tasks. For example, when you request to create a snapshot of a disk, a snapshot reference is not returned as the response but rather a ZoneOperation is returned where you can look up the status of the request including progress, start / end time, and warnings or errors. See here for more information.

I'm not yet sure what the best way to model these objects are. But for now I've assumed the default as just another object off the Zone, Region or Compute objects depending on their level.

Filters

Filter expressions are used to filter objects when listing them using the API. By reducing the barrier here, we can improve developer productivity. This means automatically encoding the request when neccesary (e.g. string has 2 spaces) however we could also assume it's already encoded if they pass in a single string with no spaces. We can also substitute 'eq' for '=='' and 'ne' for '!=' and handle all 4 options seamlessly.

Example of expression found here.

E.g. of encoding: target eq 'example-name-[0-9]+' must be encoded to filter=target+eq+%27example-name-%5B0-9%5D%2B%27 automatically for the developer but if they provide a single string with no spaces, we can assume it's already encoded.

Timestamps

Anytime there is a timestamp used in a request e.g. in Images.deprecate, we can allow the use of Number type and assume is milliseconds since last Epoch (Jan 1, 1970) or if they provide a Date object, we can convert that to a standard milliseconds timestamp.

Compute

Create Compute object

Defaults

You can provide a default zone and default region. These will be converted to Zone and Region objects on-the-fly and can be referenced in your code as compute.defaultZone and compute.defaultRegion. In addition, compute.allZones and compute.allRegions will be special objects that have a select number of functions available for performing aggregate list functions across all zones or all regions.

var compute = gcloud.compute({ zone: 'default-zone', region: 'default-region' });

var myZone = compute.defaultZone; // created automatically with default value specified
var myRegion = compute.defaultRegion; // created automatically from default value specified

var allZones = compute.allZones; // special value for aggregate functions
var allRegions = compute.allRegions; // special value for aggregate functions

Top level (Global) objects

var myZone = compute.zone('zone-name'); // See per-zone objects
var myRegion = compute.region('region-name'); // See per-region objects

var myImage = compute.image('image-name'); // done
var mySnapshot = compute.snapshot('snap-name'); // done
var myNetwork = compute.network('network-name'); // done
var myFirewall = compute.firewall('firewall-name');
var myRoute = compute.route('route-name');
var myGlobalOperation = compute.operation('operation-name');
var myBackendService = compute.backendService('backend-service');
var myForwardingRule = compute.forwardingRule('fwd-rule-name');
var myAddress = compute.address('global-addrs-name');
var myTargetHttpProxy = compute.targetHttpProxy('proxy-name');
var myHealthCheck = compute.httpHealthCheck('health-check-name');
var myInstanceTemplate = compute.instanceTemplate('template-name');
// No license object - see compute.getLicense()
var myUrlMap = compute.urlMap('urlmap-name');

Zone-level objects

var myInstance = myZone.instance('instance-name'); // done
var myDisk = myZone.disk('disk-name'); // done
var myMachine = myZone.machine('machine-type-name');
var myZoneOperation = myZone.operation('operation-name');
var myTargetInstance = myZone.targetInstance('target-instance-name');

Region-level objects

var myAddress = myRegion.address('address-name');
var myRegionalOperation = myRegion.operation('operation-name');
var myTargetPool = myRegion.targetPool('target-pool-name');
var myForwardingRule = myRegion.forwardingRule('forwarding-rule-name');
var myTargetVpnGateway = myRegion.targetVpnGateway('gateway-name');
var myVpnTunnel = myRegion.vpnTunnel('tunnel-name');

Get License

compute.getLicense('license-name', function(err, metadata) {});

Get Disk Types

// Using custom zone
compute.zone('different-zone').getDiskTypes(function(err, diskTypes, nextPageToken) {});

// Using default zone
compute.defaultZone.getDiskTypes(function(err, diskTypes, nextPageToken) {});

// Get disk types across all zones
compute.allZones.getDiskTypes(function(err, diskTypes, nextPageToken){});

Get Disk Type

// Using default zone
compute.defaultZone.getDiskType('disk-type-name', function(err, diskType) {});

// Using custom zone
compute.zone('different-zone').getDiskType({
  name: 'disk-type-name'
}, function(err, diskType) {});

Instances (per-zone)

Reference an existing Instance

// Using custom zone
var myInstance = compute.zone('different-zone').instance('my-instance');

// Using the default zone
var myInstance = compute.defaultZone.instance('my-instance');

Create a new instance in the default zone with no options

// Using default zone
compute.defaultZone.createInstance('my-new-instance', function(err, instance) {});

// Using custom zone
var options = { name: 'my-new-instance' };
compute.zone('different-zone').createInstance(options, function(err, instance) {});

Get instance metadata

myInstance.getMetadata(function(err, metadata) {
  myInstance.metadata === metadata // true
});

Set instance metadata

myInstance.setMetadata(metadata, function(err) {
  // On success, updated metadata available in myInstance.metadata
});

Get an array of instances

// Using default zone
compute.defaultZone.getInstances(function(err, instances, nextPageToken) {});

// Using custom zone
var options = { pageToken: 'next-page', filter: 'filter' };
compute.zone('different-zone').getInstances(options, function(err, instances, nextPageToken) {});

// Get aggregate list of all instances across all zones
compute.allZones.getInstances(options, function(err, instances, nextPageToken) {});

Delete the instance

myInstance.delete(function(err) {});

Reset an instance

myInstance.reset(function(err) {});

Attach a Disk to an Instance

instance.attachDisk(myDisk, function(err) {});

Detach a Disk from an Instance

instance.detachDisk(myDisk, function(err) {});

Disks (per-zone)

Reference an existing Disk (no api call)

// Using the default zone
var myDisk = compute.defaultZone.disk('my-disk');

// Using custom zone
var myDisk = compute.zone('different-zone').disk('my-disk');

Create a new Disk

var options = {
  name: 'some-disk-name',
  sourceImage: compute.image('my-image'), // or image url,
  // or
  sourceSnapshot: compute.snapshot('my-snapshot'), // or snapshot url
  // or
  sizeGb: 10 // size in GB
};

// Using default zone
compute.defaultZone.createDisk(options, function(err, zoneOperation) {});

// Using custom zone
compute.zone('different-zone').createDisk(options, function(err, zoneOperation) {});

Get an array of Disks

// optional
var options = {
  filter: 'some-filter',
  nextPageToken: nextPageToken
};

// Using default zone and no options
compute.defaultZone.getDisks(function(err, disks, nextPageToken) {});

// With options
compute.defaultZone.getDisks(options, function(err, disks, nextPageToken) {});

// Get aggregated list of disks across all zones
// Uses disks.aggregatedList() seamlessly
compute.allZones.getDisks(options, function(err, disks, nextPageToken) {});

Get Disk metadata

myDisk.getMetadata(function(err, metadata) {
  myDisk.metadata === metadata // true
});

Delete a disk

myDisk.delete(function(err) {});

Create a snapshot of Disk

// Requires an additional request to get snapshot data
myDisk.createSnapshot(function(err, zoneOperation) {});

Zones (maybe have separate resource for zones)

Get all zones

compute.getZones(function(err, zones, nextPageToken) {});

compute.getZones({
  nextPageToken: nextPageToken,
  filter: 'some-filter'
}, function(err, zones, nextPageToken) {});

Reference an existing Zone

var myZone = compute.zone('zone-name');

Get a zone

myZone.getMetadata(function(err, metadata) {
  myZone.metadata === metadata // true
});

Get all operations of zone

var options = { filter: 'filter', nextPageToken: 'nextPage' };
myZone.getOperations(options, function(err, operations, nextPageToken) {});

Get specific operation of zone

myZone.getOperation('operation-name', function(err, operation) {});

Delete a zone operation

myZone.deleteOperation('operation-name', function(err) {});

Regions

Reference an existing region (no api)

var myRegion = compute.region('region-name');

Get all regions

var options = { filter: 'filter', nextPageToken: nextPageToken }; // optional
compute.getRegions(options, function(err, regions, nextPageToken) {});

Global Operations

Get global operations

compute.getOperations(function(err, operations, nextPageToken) {});

// With options
var options = { filter: 'filter', nextPageToken: nextPageToken };
compute.getOperations(function(err, operations, nextPageToken) {});

Delete global operation

compute.operation('operation-name').delete(function(err) {});

Get global operation

var myOperation = compute.operation('operation-name');
myOperation.getMetadata(function(err, metadata) {
  myOperation.metadata === metadata // true
});

Images

Reference an existing image

var myImage = compute.image('my-image');

Create a new image

var myDisk = compute.defaultZone.disk('my-disk');
var metadata = { name: 'my-image', sourceDisk: myDisk };
compute.createImage(metadata, function(err, myImage) {

});

Get all images

var options = { filter: 'filter', nextPageToken: nextPageToken };
compute.getImages(options, function(err, images, nextPageToken) {});

Delete an image

myImage.delete(function(err, globalOperation) {});

Deprecate an image

var options = {
  state: compute.IMAGE_DELETED, // state can be 'DELETED', 'DEPRECATED', or 'OBSOLETE'
  replacement: compute.image('my-replacement'),
  deprecated: new Date()
};
myImage.deprecate(options, function(err, globalOperation) {});

Snapshots

Reference an existing snapshot

var mySnapshot = compute.snapshot('snapshot-name');

Get snapshot metadata

mySnapshot.getMetadata(function(err, metadata) {
  mySnapshot.metadata === metadata // true
});

Delete a snapshot

mySnapshot.delete(function(err, globalOperation) {});

Get list of snapshots

var options = { nextPageToken: nextPageToken, filter: 'filter' };
compute.getSnapshots(options, function(err, snapshots, nextPageToken) {});

Networks

Reference an existing network

var myNetwork = compute.network('my-network');

Get network metadata

myNetwork.getMetadata(function(err, metadata) {
  myNetwork.metadata === metadata // true
});

Get list of networks

var options = {
  filter: 'filter',
  nextPageToken: nextPageToken
};
compute.getNetworks(options, function(err, networks, nextPageToken) {});

Delete a network

myNetwork.delete(function(err, globalOperation) {});
      • Incomplete - - -
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment