Skip to content

Instantly share code, notes, and snippets.

@nateklaiber
Last active March 8, 2024 15:53
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save nateklaiber/ee068e8114d0eed09a28cd44be37c67c to your computer and use it in GitHub Desktop.
Save nateklaiber/ee068e8114d0eed09a28cd44be37c67c to your computer and use it in GitHub Desktop.
Pizza Service SDK Architecture

SDK Architecture

Rough Draft

This is still in rough draft form. I will continue to make edits as I receive them. I will collate the list of feedback here and make notes of the changelog as revisions are made.

What does it solve?

String Interpolation

No string interpolation for building your routes to your resources.

Example: Given a service URL is a pattern like: https://pizza-service/customers/{customer_id}/orders{?page,per_page,query}

// Bad
let url = apiHost + "/customers/" + customerId + "/orders";
let url = `${apiHost}/customers/${customerId}/orders`;

// Better
let url = PizzaService.Routes.urlFor('customer-orders', { customer_id: 'CUS-1234' });

Data Access

Help to guide around best practices of making requests. Eliminating arbitrarily chained promises and instead using well defined modeling and interfaces. Just as there is callback hell, there is also promise hell if not managed well.

This encapsulates the access in the objects and operations. If we see going into raw HTTP requests and responses, that should raise a flag.

Example: Given I want to access a customers pending orders, sorted descending by their created date.

// Bad
axios.get(`https://pizza-service/customers/${customerId}/orders`)
  .then(response => {
    let data = response.data;

    return data.filter((attr) => {
      return data.status_name === 'pending';
    }).order((a, b) => {
      return Date.parse(a.created_at) - Date.parse(b.created_at);
    });
  }).catch(error => {
  });

// Better
let orders = await client.models.orders.list({ customer_id: customerId }).pending().order({ created_at: 'desc' });
let orders = await client.models.orders.list({ customer_id: customerId, is_pending: true }).order({ created_at: 'desc' });

By having things encapsulated in the models, we can the benefits of collections (enumerable) and models (instances, comparable).

Another benefit here is the encapsulation can be intelligent about how it may need to perform an operation. If I need to get 'pending' orders, it may do so via:

  • API with filters: This would be the most efficient. It would use the API and proxy that request on.
  • API with pagination: This would be least efficient. It would auto-paginate and then map/reduce the result set.
  • Local with filters: This would perform the action on a local data set.

No matter what approach is taken, we keep that logic encapsulated in the model and don't leave it to the consumers to figure out (again, guidance on best practices, with flexibility to do what's needed).

The end result would be the same: the filtered collection of pending orders.

Composition

Relies on composition. Each part can be used on its own. This does not require a connection to use modeling. It does not require requests to use modeling. It builds up from the components and provides opinionated access while also giving flexibility to customize or build as needed.

By using composition, we are not tightly bound to any one part. Many SDKs require a connection or are meta-built from some other documentation output.

Example: Given I have a cached data set or a local copy of a Schema (model) and I want to use it.

// Bad
let orders = otherClient.getOrders();


// Better
let localOrdersCollection = [];

let orders = new PizzaService.Models.Orders(localOrdersCollection);

console.log(orders.totalAmountUnit());
// 34.99 USD

orders.line_items.each((line_item) => {
  console.log(line_item.name);
});

In the first example my model is tightly bound to the entire system and requires a connection to a remote service.

In the second example I can use it to interact with the core data model. I can work with the object and not inline with raw attributes. I can even load nested attributes to test the associations as needed.

Dependency / Resource Store

It is not tightly bound to any adapter. It will use Axios by default, but could also use fetch or XMLHTTP. The SDK defines the rules and interfaces and all adapters are built to that contract.

Since everything is de-coupled, it also permits you to use the raw underlying adapter if needed. You could even extend to create a FileSystem adapter that responds to the necessary interface.

// Bad
let orders = otherClient.getOrders();

// Good
let fsAdapter    = PizzaService.Adapters.retrieve('fs');
let axiosAdapter = PizzaService.Adapters.retrieve('axios');
let fetchAdapter = PizzaService.Adapters.retrieve('fetch');

// FS
let client = new PizzaService.Client({ adapter: fsAdapter });

// Will retrieve according to the underlying adapter that maps resources to FS objects. It returns a FS Response.
let orders = client.models.customers.retrieve('CUS-1234');

// Axios
let client = new PizzaService.Client({ adapter: axiosAdapter });

// Will use the Axios adapter to retrieve the data. It returns an HTTP Response
let orders = client.models.customers.retrieve('CUS-1234');

// Fetch
let client = new PizzaService.Client({ adapter: fetchAdapter });

// Will use the Fetch adapter to retrieve the data. It returns an HTTP Response
let orders = client.models.customers.retrieve('CUS-1234');

In the first example we may use directly, or have returned to us, the Axios instance. This causes an underlying dependency to bleed through to our consumers.

In the other examples we're using our interface for requests and responses and using coresponding adapters. The consumer will only ever see or interact with our models (Request/Response interfaces)

Developer Tools

It decorates and enriches the console to aid in development. This would be the equivalent of tailing a log in other services. This is important for visibility in both development and production.

It provides a namespace for the wrapped methods, permitting us to grep the console for only the things we care about.

The default logger here uses the console, but it could just as easily broadcast to other services as needed (File, Exception Service, etc)

// Bad
console.error('My error that will get included in all other errors');
// My error that will get included in all other errors

// Better
let logger = new PizzaService.Logger(console);

logger.error('My error will now include a named prefix');
// '[SERVICE_CLIENT] My error will now include a named prefix'

By wrapping the console (or any other broadcaster), we can adhere to a logging interface and tag our own package messages accordingly. This permits me to then grep for '[SERVICE_CLIENT]' or similar to only see the logs I care about.

Error Expectations

It exposes only a subset of exceptions to ensure other dependencies do not bleed through. A consumer can use the SDK with confidence.

// Bad
try {
  let orders = otherClient.getOrders();
} catch(e) {
  if(typeof e == 'ExpectedClientError') {
  }
}
// Error: uncaught exception [AxiosError]

// Better
try {
  let orders = await client.requests.orders.list();
} catch(e) {
  if(typeof e === 'ApplicationError') {
    logger.error(e);
  }
}

In the first example a developer may program around the expectation that they get a localized exception. However, the client ended up with an error from an underlying dependency being exposed. This was not accounted for. And, even if it was, it means the developer needs to know the underlying dependencies and possible exceptions that could be thrown.

In the second example, the SDK has trapped all errors in the underlying route to retrieving the orders list. The exception will always be the ApplicationError and will preserve any stack trace that's needed.

Promise Hell

Developers are often familiar with the Callback Hell (Callback Pyramid). That leads to nested callback functions within callback functions.

Now developers rely on promises. However, this can still lead to Promise Hell. The only difference is this one is flattened. But, with each chained .then you need to know what you're dealing with and manage the state from the prior promises (or indexed promises). Yes, it will read linearly, but I often challenge developers to tell me what is expected in the third chained .then in a stack.

I often see this as akin to a child saying their room is cleaned simply because they shoved everything in the closet. They didn't clean the room, they just moved it or called it by a new name.

What I care about is the objective. If I have to wait for multiple service calls to finish before being able to do things, then my blocker is the HTTP request(s).

For instance, I may place an Order. Then, once it's placed I need to immediately return a Status object. This will encompass my Deliveries and Transactions (Payments). I may need the depth of several objects to retrieve what's needed.

// Meh
let order = client.models.orders.create(orderParams)
                                .then((resolve,reject) => {
                                  reject(new Error('Oops'));
                                  resolve(return new PizzaService.Models.Order(result.asJson()));
                                }.then((order) => {
                                  return new order.getStatus();
                                }));

// Better
let orderParams = {
}
let order      = await client.models.orders.create(orderParams);
let status     = await order.status;
let deliveries = await order.deliveries;

// Or
order.with_status('pending', ((order) => {
  // Order is still pending
}));

order.with_status('payment_settled', ((order) => {
}));

order.with_status('delivery_settled', ((order) => {
}));

order.with_status('settled', ((order) => {
}));

In the first example we tightly bind all interactions to the promise chain.

In the second example we can retrieve the objects independently of one another. We can also expose interfaces to deal with things as we see fit.

/**
* This is the SDK of our fictional Pizza Service
*/
import PizzaService from '@pizza-service/sdk'
/**
* --- PACKAGE ---
* The package itself will come with tools for both the consumers and developers. This can be used as a standalone tool or
* as a part of other services/packages as needed (Express app, React, Node CLI, etc)
*
* It will have:
*
* For Developers:
* * Test suite (jest or other) that can be run for the codebase.
* * Documentation (JSDoc or other) that can be generated to visually browse the codebase
*
* For Consumers:
* Scripts to help you work with the service. Things like
*
* * yarn run pizza_service:routes - which will return the set of available routes
* * yarn run pizza_service:mime_types - which will return the set of mime type mappings. Custom and IANA.
* * yarn run pizza_service:configuration - which will output the current configuration
* * yarn run pizza_service:data:customers - which will be a CLI wrapper to interact with customers
*
* And more.
*/
/**
* --- ERRORS ---
* The SDK library will house a set of internal errors that would be raised. These errors would
* be the only errors every exposed to the consumer of the SDK. It would handle any underlying package dependencies
* that may raise errors as well. It will capture them, preserve the stack trace, and then re-throw withour decorated
* error object.
*
* Below is a list of a few of these objects.
*/
PizzaService.Errors.RecordNotFoundError;
PizzaService.Errors.RouteNotFoundError;
PizzaService.Errors.SerializerNotFoundError;
PizzaService.Errors.RequestError;
PizzaService.Errors.ConnectionError;
PizzaService.Errors.ApplicationError;
/**
* --- LOGGING ---
* The default logger is the console, but could be output to other services as needed. We setup the new logging
* models that function as proxies/delegators to `console`. We focus on 3 specific areas:
*
* * Application: This is the internal logging within the SDK itself (Business rules)
* * Cache: This would be connected to an HTTP caching service and logs hits/misses
* * Request: This would be bound to the underlying HTTP adapter (axios, fetch, etc) and would log the protocol (HTTP) communications
*
*/
const DefaultLogger = console;
let applicationLogger = new PizzaService.Logger(DefaultLogger);
let requestLogger = new PizzaService.Logger(DefaultLogger);
let cacheLogger = new PizzaService.Logger(DefaultLogger);
/**
* --- CONFIGURATION ---
* The configuration object is the root holder of all of the configuration options. These include
* things like API service information, default headers, default query string parameters, connection
* settings, and internal logging.
*
* It can be instantiated via its constructor.
* It can be instantiated via a #configure block after initialization.
*
*/
const configurationParams = {
host: process.env.PIZZA_SERVICE_HOST,
token: process.env.PIZZA_SERVICE_TOKEN,
oauth: {
client_id: process.env.PIZZA_SERVICE_CLIENT_ID,
client_secret: process.env.PIZZA_SERVICE_CLIENT_SECRET,
},
headers: {
user_agent: PizzaService.Client.getName()
},
logger: {
application: applicationLogger,
cache: cacheLogger,
request: requestLogger
}
}
let configuration = new PizzaService.Configuration(configurationParams);
let configuration = PizzaService.Configuration.configure((config) => {
config.setHost(process.env.PIZZA_SERVICE_HOST);
config.setToken(process.env.PIZZA_SERVICE_TOKEN);
});
/**
* --- CLIENT ---
* The client is the root object that will coordinate communication between other resources,
*
* It can be instantiated via its constructor.
* It can be instantiated via static method that takes in a configuration object
* It can be instantiated via a #configure block after initializations
*
*/
let client = new PizzaService.Client(configurationParams);
let client = PizzaService.Client.fromConfiguration(configuration);
// This would merge options that are already set
client.configure((config) => {
config.setHost('https://new-host');
}:
// Usage would function like this.
console.log(client.getHost());
console.log(client.getToken());
console.dir(client.loggers.retrieve('application'));
console.dir(client.getDefaultHeaders());
// Show a table of all mime types
console.table(client.mime_types.list().toDataTable());
/**
* --- CONNECTION ---
* The connection is the root object that handles the communication with the underlying service.
*
* The params here may come directly from our root configuration object.
*
* By default this simply prepares the connection to be used. It is not typically invoked by itself,
* but will be used for requests.
*
* This may throw a PizzaService.Errors.ConnectionError.
*/
const connectionParams = {
headers: {
user_agent: 'Override',
},
timeout: 1000
}
let connection = new PizzaService.Connection(connectionParams);
let connection = client.connection:
/**
* --- ROUTES --
* Routes will be our dictionary lookup to our underlying resources. The routing library functions alongside
* the protocol adapter (HTTP as default).
*
* If a service exposes a root endpoint (Directory, Phone Book) - we can use that
* We can specify our own routing library via a `routes.json` file
*
* The goal here is to not have to rely on the many ways SDKS use string interpolation or other merge tactics
* with a URI. This keeps things uniform.
*
* By default these say nothing about the underlying resources or representations. This is simply a way to
* get the URI.
*
* This may throw a PizzaService.Errors.RouteNotFoundError
*/
let routes = await PizzaService.Routes.list();
let routes = await client.routes.list();
// Here is an example that uses a URI template
let route = await client.routes.retrieve('rels/customer');
// <Route href: 'https://pizza-service.com/{id}'>
let url = route.urlFor({ id: 'CUS-123' });
// https://pizza-service.com/CUS-123
/**
* --- REQUEST ---
* Requests will be communicating over HTTP, but with this architecture we could communicate
* via other protocols as well (FS, SFTP, FTP, etc). Our #connection knows how to connect to the service
* and the #request knows the adapter type to be used.
*
* By default we will have an Axios Adapter, which handles its own Request and Response
*
* adapters/axios/request.js
* adapters/axios/response.js
*
* This would come packaged with the SDK so there's an immediate way to perform HTTP operations.
*
* We then have our own Request model that delegates to the underlying adapter and exposes all base
* operations we need (GET/PUT/POST/PATCH/DELETE/HEAD/OPTIONS).
*
* An important note here is the request is only responsible for requesting and returning a response. It
* makes no judgement on the protocol level errors (4XX, 5XX). It issues a request and returns the protocol (HTTP)
* response.
*
* Many of the options this takes can come directly from our root configuration objecet or can be modified at runtime.
*
* This may throw a PizzaService.Errors.RequestErrror
*/
let adapter = client.configuration.getDefaultAdapter();
let adapter = client.getDefaultAdapter();
let adapter = PizzaService.Configuration.getDefaultAdapter();
const requestParams = {
base_url: process.env.PIZZA_SERVICE_URL,
headers: configuration.getDefaultHeaders(),
}
let httpRequest = new PizzaService.RequestTypes.Http(requestParams);
// These examples show raw requests. Within our SDK we would instead reply on the objects for Routes and Mime Types mappings. The first example
// shows these variations
let getRequest = await httpRequest.get('/resource');
let route = await client.routes.retrieve('rels/resource');
let url = route.urlFor();
let mimeType = client.mime_types.retrieve('application/json');
let getRequest = await httpRequest.get(url, { content_type: mimeType });
let postRequest = await httpRequest.post('/resource', { content_type: 'application/json' }, { data: { name: 'testing' } });
let putRequest = await httpRequest.put('/resource/123', { content_type: 'application/json' }, { data: { name: 'testing' } });
let patchRequest = await httpRequest.patch('/resource/123', { content_type: 'application/json' }, { data: { name: 'testing' } });
let headRequest = await httpRequest.head('/resource/123');
let optionsRequest = await httpRequest.options('/resource/123');
// These are also available from within the client itself
let getRequest = await client.requests.get('/resource');
let postRequest = await client.requests.post('/resource', { content_type: 'application/json' }, { data: { name: 'testing' } });
// We will then have very specific requests mapped to our Service Resources. Their job is to utilize our Routes and handle all
// request interactions with that specific Resource/Representation.
//
// These will later be used to join/map to the actual Domain Model
let customersRequest = await PizzaService.Requests.Customers.list();
let customersRequest = await client.requests.customers.list();
let customersRequest = await client.requests.customers.list({ accept: 'text/csv' }, { per_page: 100 }, ((req) => {
req.on('success', ((resp) => {
console.log(resp.body().decoded());
});
})));
/**
* --- RESPONSE ---
* Responses will be wrappers around the underlying adapter response (based on type like HTTP or other). Our default approach
* would be HTTP and the response would include the standard http response:
*
* Headers
* Body (Raw)
*
* This will also preserve raw access to the issued request from the HTTP adapter for reference.
*/
let response = await httpRequest.get('/resource/123');
// Once we have a response, then we have helper methods in our native response for handling
// given the different scenarios. This is preferred over static checking of status codes, as we may be dealing
// with an adapter that is not HTTP. With this, we can use this same response (or subset) interface for the FileSystem or
// another protocol (SFTP, FTP).
console.log(response.isSuccess());
console.log(response.isRedirect());
console.log(response.isFailure());
console.log(response.isServerError());
// These are equivalent to the above if you want to use if/then/switch statements. This wraps it with a function. This
// is not meant to be a chainable Promise or anything else, it's simply a way to only execute if the `on` state maches.
response.on('success'), ((resp) => {
}));
response.on('redirect'), ((resp) => {
}));
response.on('failure'), ((resp) => {
}));
response.on('server_error'), ((resp) => {
}));
// By default the response body is the raw value of the response (String/IO). In the case of our HTTP adapter, we can
// use content negotiation to perform helper typecasting.
console.log(response.body());
console.log(response.body().asJson());
console.log(response.body().asJsonDataTable());
/**
* --- SERIALIZERS ---
* Serializers will be defined for the expected schemas. For our default HTTP service we will be able to map them to their
* corresponding Accept/Content-Type headers (via MIME type mappings).
*
* Separating this concerns means we can use these independently of the source of the data. These could be their own packages as
* they have other binding into this system. Where our service has a specific schema, we can define a serializer and add it to the
* base package definition.
*/
let serializer = PizzaService.Serializers.retrieve('application/vnd.pizza.json_data_table');
let serializer = client.serializers.retrieve('application/vnd.pizza.json_data_table');
console.log(response.body().serializedAs(serializer));
console.log(serializer.read(response.body()));
// When using an adapter that has the corresponding MIME type mappings and we can detect it, we can return
//
// * The raw response
// * The default decoded response from the serialzied mapping
// * Any serializer specified
//
// In this example, the client sent in an Accept header of 'text/csv' and desires to receive that. The response
// has the raw body(), and then calling .decoded() will use the corresponding mime type mapping to serialize the response.
let customersRequest = await client.requests.customers.list({ accept: 'text/csv' }, { per_page: 100 }, ((req) => {
req.on('success', ((resp) => {
console.log(resp.body().decoded());
});
})));
/**
* --- MODELS ---
* Models are a very simple definition that relate to Collections and Models (Instances) of Resources.
*
* These models are meant to be used as standalone with simple arguments of an Array or Object.
* The models can be used independently of how the data gets sourced (HTTP, Filesystem, etc)
* The models handle an incoming schema and know how to handle/typecast the attributes and relationships
*
* The typical Connected Model will then be composed of the SDK parts:
*
* * The client.connection: Connect to the service
* * The client.requests: Will utilize the underlying connection and request adapter.
* * The client.routes: Will utilize the underlying routes to find for specified resource
* * The Response: Will utilize the response from the service
* * Serialization: Will utilize HTTP Content negoitation and serializers.
*
* This may throw a PizzaService.Errors.ApplicationError
*/
// Here is an example implementation within a Connected Model
// This is CustomersRequest.list().
// This can be composed hwoever is needed. The trivial example here also includes extended logging and timing
async list(attributes) {
let route = this.routes.findByRel('rel/customers');
let url = route.urlFor((attributes || {}));
try {
this.requestLogger.info(`Issuing request to ${url}`);
let timer = `Starting request to ${url}`;
this.requestLogger.time(timer);
let request = await this.adapter.get(url);
this.requestLogger.timeEnd(timer);
return request;
} catch(err) {
this.requestLogger.error(`Error with request: ${err}`);
throw new RequestError(err);
}
}
// This is the ConnectedCustomers.list(). This method is safe in that it will swallow errors and log them. There can
// also be variants/arguments that will throw underlying exceptions. It's up to us.
//
// This is a trivial example and is flexible to be composed however is desired (Composition).
async list(attributes) {
let records = ConnectedCustomers.none();
try {
this.applicationLogger.info(`Retrieving model for Customers`);
let request = await this.requests.customers.list(attributes);
request.on('success', ((resp) => {
records = new ConnectedCustomers(resp.body().asJson());
}));
request.on('failure', ((resp) => {
this.applicationLogger.error(`Error retrieving customers: ${resp}`);
}));
request.on('server_error', ((resp) => {
this.applicationLogger.error(`Error retrieving customers: ${resp}`);
}));
} catch(err) {
this.applicationLogger.error(`Error retrieving customers due to exception: ${err}`);
}
return records;
}
let customers = await client.models.customers.list();
console.table(customers.toDataTable());
// Here we can use with a basic Array structure
const customersParams = [
{
id: 'CUS-1213',
first_name: 'Lester',
last_name: 'Tester',
birth_date: '1945-03-12'
}
]
let customers = new PizzaService.Models.Customers(customersParams);
console.log(customers.toDataTable());
customers.each((customer) => {
console.log(customer.getFullName());
// This would return a Date object
console.log(customer.getBirthDate());
// This would return the corresponding date math to show their age
console.log(customer.getAge());
// This would return the underlying associations.
console.log(customers.orders().any());
console.log(customers.orders().count());
}):
// We can also use this via our data store from our underlying Adapter. An important thing to
// note about this modeling. This will return a ConnectedModel, which is the union of our `client` (Connection) and
// our static modeling. This will handle the interactions to make the request to retrieve the data
// and then instantiate our static model. It will override where needed (associations) and delegate to
// the static model where not needed.
let customers = await client.models.customers.list();
console.log(customers.toDataTable());
customers.each((customer) => {
console.log(customer.getFullName());
console.log(customers.orders().any());
console.log(customers.orders().count());
}):
// These models will often return builders or an instance of themselves, permitting us to do things like
let customer = await client.models.customers.retrieve('CUS-1234');
// Only retrieve pending orders for this week
let orders = customer.orders().where({ date_range: 'this_week' }).pending.sortBy({ created_at: 'desc' };
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment