Skip to content

Instantly share code, notes, and snippets.

@chelseasanc
Last active October 2, 2015 15:18
Show Gist options
  • Save chelseasanc/379f9788cde190a6b173 to your computer and use it in GitHub Desktop.
Save chelseasanc/379f9788cde190a6b173 to your computer and use it in GitHub Desktop.

Table of Contents

What is a Channel?

A Channel is an interface that communicates with external APIs. The goal of a Channel is not to be a direct interface that sits on top of an API, but instead a user-friendly abstraction on top of an API. Channel development is more focused on end-user needs than on the capabilities of the API. Each Channel is made up of methods, which appear to the user as the different Event and Action cards. Each method determines how data is fetched from the API, and transforms data to the user-friendly format accepted by the front-end. Most methods use more than one API call to do this.

To define a Channel, you must write a Channel JSON file and submit this file to Azuqua for upload into the engine. At runtime, the engine will access the instructions laid out in this file to execute the Event or Action the user has designated in their Flõ. The primary function of a Channel JSON file is to lay out in a linear manner the pre-defined action steps (known internally as bricks) that will execute each user-facing event or action. Using bricks, you can define what data will be fetched from the API, select the data you want to pass on to the user, and transform this data into flat JSON that can be consumed by the Azuqua front-end.

General Channel Structure

{
	  "name": "Example Channel",
	  "description": "Example Flõ Channel",
	  "version":"0.0.3",
	  "type": "channel",
	  "recurrence": 1,
	  "realtimeavailable": true,
	  "dependencies": {
		// Custom code dependencies
	  },
	  "auth": { 
	  	// Auth object
	  },
	  "methods": [
		//An array of methods, described in the #Methods Section
	  ]
	}

Fields:

  • name, the name of your Channel. The name musst be in title case, as it will appear in the UI
  • description, the description of your Channel. Not required, but may be used for internal purposes
  • version, the version number of this Channel. Follows Semantic Versioning(http://semver.org/) specifications
  • type, must be set to "channel"
  • recurrence, must be set to "1"
  • realtimeavailable, boolean value that if true, indicates that the Events for this Channel are webhooks (do not need to be scheduled like a polling Event). If no value given, defaults to false.
  • dependencies, expects a dependencies(#custom-code-dependency) object
  • auth, expects an auth schema(#authorization) object

Authorization

Azuqua supports 3 Auth schemes: basic, OAuth (both 1 and 2) and custom.

Basic Auth

"auth": {
	"type": "basic",
	"authparams": {
	  "username": {
		"type": "string",
		"displayname": "Username"
	  },
	  "password": {
		"type": "password",
		"displayname": "Password"
	  },
	  "instance_url": {
		"type": "string",
		"displayname": "Instance URL (w/o https)"
	  }
	}
  },

A basic auth object can take any auth parameters that the user defines. Each object in the authparams object must have a type (options: string, password) and a displayname (will be shown in the UI, must be in title case). You can access the user inputs for each of these authparams using mustache(#azuqua-mustache-rules). The only difference between basic and custom authorization schemes is the ability to use certain convenience fields, such as the auth field in the HTTP brick(#http). If you use basic authorization, using mustache to enter your authorization credentials into the auth field will automatically add them to your request as an Authorization header. If you use custom, you must explicitly build the Authorization header yourself.

Custom Auth

"auth": {
	"type": "custom", 
	"authparams": {
	  "username": {
		"type": "string",
		"name": "Username",
	  },
	  "password": {
		"type": "password", 
		"name": "Password",
	  }
	}
  },

Like basic auth, the custom auth object can take any auth parameters the user defines, as long as they have both a type and a name. When using this auth scheme, you cannot use the auth parameter in the HTTP brick(#http).

Methods

The majority of the Channel body is dedicated to describing the Channel's methods. There are three kinds of methods: Events, Actions, and metadata methods.

Event and Action methods will be uploaded into the UI and represented as cards. Event methods are the cards that monitor external applications and start FLOs when the triggering event occurs. Action methods are the cards that take action in an application in response to the starting event of the FLO.

Helper methods work under the hood, running processes that supplement the main process of the Event or Action. Metadata methods are used to handle collections, generate dynamic dropdowns, and more.

All of the methods associated with a channel are stored in a methods array as shown in the channel file structure.

Method Object Structure

Each method is an object with the following structure:

{
 "name": "Issue Created", //The name of the method. This cannot be changed, so be sure to consult the style guide for any methods that will appear in the UI.
	 "description": "A new issue has been created", //This will also appear in the UI, but it can be changed. 
	 "kind": "event", //Other options include 'action' and 'metadata'
	 "params":[
		//an array of parameter objects
 ],
 "input": {
	//an input object
 },
	 "output": {
	//an output object
 },
	 "zebricks": [
	//an array of brick objects
 ]
}		 

Params Object

The params object defines any parameters that users will need to fill out at run-time in order to properly build the card or make the card more convenient for the user. In some cases, parameters help provide information that allows Azuqua to dynamically build cards that include users' custom fields.

There are three kinds of parameter objects that you can use in your parameters array:

Field Parameter

This parameter object defines a text input field that the user will have to type into to complete card setup.

The search field in the Twitter "Monitor Keyword" card lets users define the search terms they would like to monitor before exposing the Output data on the card.

Input Object

This object defines the input fields on your card. This is the basic structure of the input object:

"input": {
		"extensible" : false, //Set to true if you want users to be able to manually add input fields. Only use this option if you cannot dynamically generate custom fields from user's records, since users will have to enter fields exactly as they appear in the API. 
		"attributes" : [
			{
				"name": "header", //A header on the card. Each input object must have at least one, although you can define multiple headers order to organize your input fields.
				"attributes": [
					{
						"name": "field1", //the name of your field
						"type": "string" //must always be set to string
					},
					{
					"name": "field2",
					"type": "string"
				}
				]
			}
		]
}

This information is not only used by the front end to build the card, it also provides a schema for building a JSON object from the users' inputs. This object will then be passed to the bricks array and processed.

For example, if you want to pass the following JSON object into your bricks:

{
	"User": { 
		"firstName": "Demo", 
	"lastName": "Lovato",
	"email": "demo@unicorn.com"
	},
	"Company": {
		"name": "Unicorn Industries",
		"revenue": "1000000000"
	}
}

..your input object would look like this:

"input": {
 "extensible": false, 
 "attributes": [
	{ 
      "name": "User",
      "attributes": [
        {
          "name": "firstName",
          "type": "string"
        }, {
          "name": "lastName",
          "type": "string"
        }, {
          "name": "email",
          "type": "string"
        }
      ]
    }, {
	  "name": "Company",
      "attributes": [
        {
          "name": "name",
          "type": "string"
        }, {
          "name": "revenue",
          "type": "string"           
        }
	  ]
	}
	  ]
}

Output Object

This object defines the draggable output data that your card will return. This is the basic structure of the output object:

"output": {
		"extensible" : false, //Set to true if you want users to be able to manually add output fields. Only use this option if you cannot dynamically generate custom fields from user's records, since users will have to enter fields exactly as they appear in the API. 
		"attributes" : [
			{
				"name": "header", //A header on the card. Each output object must have at least one, although you can define multiple headers order to organize your output fields.
				"attributes": [
					{
						"name": "field1", //the name of your field
						"type": "string" //must always be set to string
					},
					{
					"name": "field2",
					"type": "string"
				}
				]
			}
		]
}

This information is not only used by the front end to build the card, it also provides a schema for the JSON object that will result from the zebricks array. If the zebricks array does not create an object that matches this schema, the method will not work.

For example, if the result of your zebricks array will look like this:

{
	"Post": { 
		"Text": "Bagels are just donuts without the happiness.", 
	"URL": "www.donutfans.com/post/1029384756",
	"Reblogs": "47"
	},
	"Author": {
		"Username": "KrullerKing",
		"Followers": "189"
	}
}

...your output object would look like this:

"input": {
 "extensible": false, 
 "attributes": [
	{ 
      "name": "Post",
      "attributes": [
        {
          "name": "Text",
          "type": "string"
        }, {
          "name": "URL",
          "type": "string"
        }, {
          "name": "Reblogs",
          "type": "string"
        }
      ]
    }, {
	  "name": "Author",
      "attributes": [
        {
          "name": "Username",
          "type": "string"
        }, {
          "name": "Followers",
          "type": "string"           
        }
	  ]
	}
	  ]
}

Zebricks Array

The Zebricks array holds a list of the steps you will take to call the API and transform the resulting data into something that the Azuqua front-end can handle.

If the card has an input object, data is passed to the bricks array according to the schema defined in the input object. The data that results from the bricks array must conform to the shape defined in the output object. The input and output objects are used to build the card at design time, so users can define how they want the different cards to handle data even before the call to the API has been run. The zebricks array handles the actual data exchange at runtime, so if the data that results from the array does not conform to the the output object, the right parameters will not be sent on to the next card and the FLO will break.

You can use Mustache templates to hash data from the input into different bricks, or use the bricks to transform the raw data.

Read more about the individual bricks you can use in the Bricks Library section.

Mustache

To hash user inputs into the bricks, Azuqua uses Mustache templates (you can read more about Mustache templates here: https://mustache.github.io/mustache.5.html).

You can see examples of how to use mustache in bricks in the Bricks Library.

Custom Code

Although most interactions with RESTful APIs can be handled with the prebuilt set of bricks, it is sometimes necessary to write custom JavaScript code to manipulate the data returned from an API.

Using the "custom" brick, you can call custom code from your bricks array and store the results in the prevData array. All of your custom code should be contained in a file called index.js.

Using the Channel Builder tool, you can paste in custom code you've written to test your channel. It is strongly recommended that you test your code before pasting it into the Channel Builder, since a major error in your code may cause you to lose progress.

Working with prevData and allData

Internally, Azuqua stores the result of each brick in its own array, allData. With the exception of the HTTP brick, each brick is implicitly modifying the output of the previous brick (prevData) and storing the result in the array.

However, you can explicitly call a specific element of prevData as part of a brick using mustache. In the example below, the second HTTP call is using a property of the response from the first HTTP call.

[
	{
		"brick": "http",
		"config": {
			"method": "GET",
			"url": "www.notarealapp.com/api/getObject?objecttype=project"
		}
	}, {
		"brick": "http",
		"config": {
			"method": "POST",
			"url": "www.notarealapp.com/api/createTask"
			"body": {
				"project_id": "{{prevData.project_id}}"
			}
		}
	}
]

You can use the same method to hash in an element in from allData array, but you must use the index of the brick whose results you are trying to call. For example, if you want to hash a property from the second brick in the brick array, you would use {{allData.1.property_name}}.

Bricks Library

Bricks are the built-in utilities you can use to handle API interaction, data manipulation, collections, and more. Each method uses an array of bricks to take input data from the user (if the method is an Action), use this data to run a call to the API, and transform the results into something the front-end can consume. Read more about the bricks array here.

In this section you can access documentation, templates, and examples for each type of brick.

Collections

The collections brick lets you run a series of operations on arrays of data. This brick is especially useful when you are building Events and querying an API endpoint for a list of new results. The collections brick takes in prevData (to use collections, prevData must contain an array) and will return the modified array.

The basic structure of the collections brick looks like this: { "brick": "collections", "config": { "operation": "some operation" //the name of the operation you want to perform ... //additional config properties specific to this operation } }

There are 4 collections operations you can use: map, filter, flatten, and limit. There are two other bricks that work with arrays but are not part of the collections operations: Massage and Sort. You can use Massage to transform objects in arrays, and you can use sort to sort arrays. Read more about each operation:

Map

This brick runs another brick operation on each element of an array, replacing that element with the result of the brick operation.

In the example brick below, the brick is iterating over the prevData array. For each item record in that array, collections is calling the http brick using the key record.id in the URL.

{
	  "brick": "collections",
	  "config": {
		"operation": "map",
		"item": "record",
		"call": {
		  "brick": "http",
		  "config": {
			"method": "GET",
			"url": "{{{auth.instance_url}}}/api/readrecord/{{record.id}}",
			"headers": {
			  "Authorization": "Bearer {{auth.access_token}}"
			}
		  }
		}
	  }
	}

Custom

Allows a way to attach custom code to a brick - the custom code lives in modules/definition/{channelName}/custom/index.js.

Do not use this when writing a channel. Also, third parties are not allowed to use this due to XSS concerns (the upload script disallows any custom bricks).

The custom code must be entered as a dependency in the channel's dependencies field.

This brick is used for calling internal Azuqua functions (deprecated), or calling another language from a channel.

Examples: s3, smartsheet, mandrill

The custom zebrick will call whatever method is in its config property. That method takes an options argument and a callback. As elsewhere, mustache is fair game (make sure to utils.render them in the custom source!)

NB. utils.render takes an object, not a string (compare mustache.render) { "brick": "custom", "config": { "method": "doThisThing", "arbitraryProprtyOne": "{{input.org_id}}" } }

And the custom code itself looks something like this:

	exports.doThisThing = function(options, callback) {
	  // Get the mustache'd property from config
	  // (utils.render takes an object and a context- usually `options`)
	  var org_id = utils.render(options.config.arbitraryProprtyOne, options);

	  internalMethod(org_id, callback);

Dateslice

The Dateslice brick takes a list sorted by the numeric value stored at path and slices off all records where the path value is less than the {{since}} value saved by the engine. This brick is used to ensure that FLOs only run with new data, but it should only be used when it is not possible to put the {{since}} value directly into the query of the call to the API to retrieve fresh data.

This brick assumes prevData (that is, the most recent brick result) is a list is in reverse date-sorted order.

	{
	  "brick": "dateslice",
	  "config": {
		"date": "{{since}}"
		"path": "date_created"
	  }
	}

Hash

The hash brick uses a key (that can either be hard-coded or mustache) to look up a value in a hash.

{
		"brick": "hash",
		"config": {
			"key": "a string or a {{mustache}} template",
			"hash": {
				"a": "hello",
				"b": "world"
			}
		}
}

HTTP

{
  "brick": "http",
  "config": {
	"method": "POST", //Takes any valid HTTP verb
	"format": "urlencoded", //Convenience property that sets the content-type header and encodes the body. Other options:json,xml,plain 
	"url": "{{{auth.subdomain}}}/api/search",
	"auth": "{{auth.username}}:{{auth,password}}", 
	"query": { //will be appended to url after ?    
	}                  
	"header": {  
	}                  
	"body": {
	}
  }
}

This is the core brick. This brick implements the HTTP protocol, which is used in every REST API.

This brick's config expects two required fields, url and method. You can hard-code a url, or use mustache to insert authorization data taken from the user at design time (as shown above).

Optionally, you can specify other HTTP fields: query, body and headers. All fields in query will be appended to the url after a ? (for example, a field named search will appear in the URL as ?search=)

  • auth, can be set to false (if your auth scheme is OAuth) or be used to contain your auth object (if your auth scheme is basic) which will automatically be transformed into Base64 and sent as a header in your HTTP request.

These is also a boolean auth field, which- if false- disables the engine from automatically performing oAuth token exchange before the request is sent. This is only necessary if your channel's top-level auth property is oauth.

An example (from the Marketo channel):

"brick": "http", "config": "method": "POST", "url": "{{auth.marketo_endpoint}}/identity/oauth/token", "auth": false, "query": "grant_type": "client_credentials", "client_id": "{{auth.client_id}}}", "client_secret": "{{auth.client_secret}}}" } } }

Which will return the raw response, which in this case is a JSON object {"access_token": "beadgcf_0"}. In the next brick, you could call {{prevData.access_token}} to access that value.

Massage

Massage changes the structure of JSON, allowing you to take the output of an API and transform it into data that the front-end can use.

This, along with the HTTP brick, implements most of the functionality you need to build a channel.

Examples: Marketo, DnB

This brick's config expects a single element, schema.

{
		"brick": "massage",
		"config": {
			"schema": {
				"type": "object" //other options: array
				"properties": {
					"Headeader" : {
						"type": "object",
						"properties": {
							"Field 1": {
								"type": "string",
								"path": "api.response.param_1"
							},
							"Field 2": {
								"type": "string",
								"path": "api.response.param_5"
							}
						}
					}
				}
			}
		}
}

This object defines a JSON schema (http://json-schema.org/) that will transform your output

{
  "brick": "massage",
  "config": {
	"schema": {
	  "type": "array",
	  "items": {                
		"type": "object"        
		"properties": {         
		  "userEmail": {       
			"type": "string",
			"path": ["email"]   
			},                  
		  "firstName": {
			"type": "string",
			"path": ["name.first"] 
		  }                       
		}
	  }
	}
  }
}

This example will take data that looks like this:

[{
		email: lito@azuqua.com,
		name: {
			first: "Lito",
			last:  "Nicolai"
	}
}, {
	  email: fake@azuqua.com,
	  name: {
		first: "Urist",
		last:  "McFakerson"
		}
}]

And transform it into something that looks like this:

[{
		email: lito@azuqua.com,
		firstName: "Lito"
	}, {
		email: fake@azuqua.com,
		firstName: "Urist"
}] 

Note how it applied the schema across the array.

Parse

Parse takes a string of plaintext and parses it into JSON. It's useful for APIs that only return plaintext responses.

Brick template: { "brick": "parse", "config": { "JSONString": "(either a string or a mustache template)" } }

Scope

Warning: If this brick is passed data where the most recent result is not an object, this brick will return null.

Grabs a single field of the incoming data. For instance, if the response is an object with an array property results, you could grab just the first element of results with { "brick": 'scope', "config": { "path": "results.0" } }

Since

{ 
	"brick" : "since",
		"config" : {
		"path": "", //
		"format": "",
		"invalid": [
		]
	}
}

A since brick should be present in every Event method (except in the case of a webhook-driven event). This brick can be configured to continue only if there was any change in an external resource, so FLOs can be triggered by changes in external resources.

field indicates which property of the incoming JSON object the time should be read from.

The format describes what format the time is read as. It can be

  • unix
  • iso8601
  • iso8601+0000
  • yyyy-MM-dd HH:mm
  • yyyy-MM-dd HH:mm:ss

Wrap

Wraps the incoming data in an object. { "brick": 'wrap', "config": { "in": "results" } }

Channel Style Guide

When you are building a Channel, choosing the corre

Event Names

Event names should describe the event that will trigger the FLO. The basic formula is adjective (e.g. new, updated, deleted) + record type (e.g. contact, post, lead).

Examples:

  • New Tweet from Location
  • Updated Lead
  • Deleted Contact

Event names should be as specific as possible. In the first example above, the name indicates that this FLO will not be triggered by all new tweets, but by new tweets from a specific location.

Event names should not contain verbs. For example, "Monitor Tweets" is not a good Event name, since it isn't describing the Event that will trigger the FLO.

Action Names

Action names should describe

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment