A request comes in to the server.
Underneath-the-hood, that request gets normalized into a Query object.
A Resource configures how to resolve a Query when given a base Scope. A scope can be any object - from an ActiveRecord::Relation to an HTTP client to an empty hash.
The user may request a resource and its relationships in one API call. This is called sideloading. Resources are re-used for sideloading, sideloading simply defines the relationships between Resources.
Once we've resolved the query into objects, we want to serialize those objects as JSON (which may include formatting/display logic). We use jsonapi-rb for this; the API is very similar to the currently-in-use active_model_serializers.
Our base application is an employee directory. Here we'll set up index (list multiple entities) and show (single entity) endpoints for an Employee model.
Code:
class EmployeesController < ApplicationController
jsonapi resource: EmployeeResource
def index
render_jsonapi(Employee.all)
end
def show
scope = jsonapi_scope(Employee.where(id: params[:id]))
render_jsonapi(scope.resolve.first, scope: false)
end
endclass EmployeeResource < ApplicationResource
type :employees
endclass SerializableEmployee < JSONAPI::Serializable::Resource
type :employees
attribute :first_name
attribute :last_name
attribute :age
endTests:
RSpec.describe 'v1/employees#index', type: :request do
let!(:employee1) { create(:employee) }
let!(:employee2) { create(:employee) }
it 'lists employees' do
get '/api/v1/employees'
expect(json_ids(true)).to eq([employee1.id, employee2.id])
assert_payload(:employee, employee1, json_items[0])
end
endRSpec.describe 'v1/employees#show', type: :request do
let!(:employee) { create(:employee) }
it 'returns relevant employee' do
get "/api/v1/employees/#{employee.id}"
assert_payload(:employee, employee, json_item)
end
endA note on testing: these are full-stack request specs. We seed the database using factory_girl, randomizing data with faker, then assert on the resulting JSON using spec helpers.
One line of code allows simple WHERE clauses. If the user tried to filter on something not whitelisted here, an error would be raised.
https://github.com/richmolj/employee_directory/compare/master...step_1_add_filter
Sometimes WHERE clauses are more complex, such as prefix queries. Here we'll query all employees whose age is greater than or equal to a given number.
https://github.com/richmolj/employee_directory/compare/step_1_add_filter...step_2_add_custom_filter
Sorting comes for free, but here's a test for it. We would decide as a team if we actually need to write a spec here, or if it's considered tested within the libraries.
Sometimes we need more than a simple ORDER BY clause, for example maybe we need to join on another table. In this example, we switch from Postgres's default case-sensitive query to a case in-sensitive one...but only for the first_name field.
https://github.com/richmolj/employee_directory/compare/step_3_basic_sorting...step_4_custom_sorting
Pagination also comes for free, so once again we'll have to decide if writing a spec like this is worth the bother.
https://github.com/richmolj/employee_directory/compare/step_4_custom_sorting...step_5_pagination
By default we use the Kaminari library for pagination. This shows how we could instead sub-out Kaminari and replace it with will_paginate
https://github.com/richmolj/employee_directory/compare/step_5_pagination...step_6_custom_pagination
For default statistics, (count, sum, average, maximum and minimum), simply specify the field and statistic.
https://github.com/richmolj/employee_directory/compare/step_6_custom_pagination...step_7_stats
Here we add a median statistic to show non-standard custom statistic usage.
https://github.com/richmolj/employee_directory/compare/step_7_stats...step_8_custom_stats
Let's say we wanted the employee's age to serialize Thirty-Two instead of 32 in JSON. Here we use a library to get the friendly-word doppleganger, and change the test to recognize this custom logic.
https://github.com/richmolj/employee_directory/compare/master...custom-serialization
Get employees and their positions in one call.
https://github.com/richmolj/employee_directory/compare/master...step_9_has_many
Get employees, positions, and the department for those positions in one call:
https://github.com/richmolj/employee_directory/compare/step_9_has_many...step_10_belongs_to
In this example an Employee has many Teams and a Team has many Employees.
https://github.com/richmolj/employee_directory/compare/step_13_error_handling...many-to-many
In prior steps we created PositionResource and DepartmentResource. These objects may have custom sort logic, filter whitelists, etc - this configuration can be re-used if we need to add /api/v1/positions and /api/v1/departments endpoints.
https://github.com/richmolj/employee_directory/compare/step_10_belongs_to...step_11_resource_reuse
This comes for free. As long as the associated Resource knows how to do something, we can re-use that logic.
In this example we add global error handling, so any random error will return a JSONAPI-compatible error response. Then we customize that response for a specific scenario (the requested employee does not exist).
Basic example without validations or strong parameters.
https://github.com/richmolj/employee_directory/compare/bump_gemfile_for_writes...step_14_create
Validations are basic, vanilla Rails code. When there is a validation error, we return a jsonapi-compatible error respone.
https://github.com/richmolj/employee_directory/compare/step_14_create...step_15_validations
The biggest problem with strong_parameters is that we might want to create an employee from the /employees endpoint, or we might want to create a position with an employee at the same time from /positions. Maintaining the same strong parameter hash across a number of places is difficult.
Instead we use strong_resources to define the parameter template once, and re-use. This has the added benefit of being built on top of stronger_parameters, which gives us type checking and coercion.
Note: strong_resources requires Rails.
Looks very similar to create.
More or less basic Rails.
https://github.com/richmolj/employee_directory/compare/step_17_basic_update...step_18_basic_destroy
So far we've shown ActiveRecord. What if we wanted to use a different ORM, or ElasticSearch? What if we wanted 'side effects' such as "send a confirmation email after creating the user"?
This code shows how to customize create/update/destroy. In this example we're simply logging the action, but you could do whatever you want here as long as you return an instance of the object. Just like with reads, if any of this code becomes duplicative across Resource objects you could move it into a common Adapter.
Think Rails' accepts_nested_attributes_for, but not coupled to Rails or ActiveRecord. Here we create an Employee, a Position for the employee, and a Department for the position in one call. This is helpful when dealing with nested forms!
Once again, note how our strong_resources can be shared across controllers.
We got this for free, here's a spec!
We get this for free, though we have to explicitly tell strong_resources that destroys are allowed from this endpoint.
Note destroy will do two things: delete the object, and make the foreign key on the corresponding child in the payload null.
destroy actually deletes objects, what if we want to simply disassociate the objects by making the foreign key null? We get this for free, too.
Let's say the departments come from a BAS call. Here's the change to the /departments endpoint:
class DepartmentsController < ApplicationResource
jsonapi resource: DepartmentResource
def index
- render_jsonapi(Department.all)
+ render_jsonapi({})
end
endclass DepartmentResource < ApplicationResource
type :departments
+ use_adapter JsonapiCompliable::Adapters::Null
+ def resolve(scope)
+ Department.where(scope)
+ end
endDepartment.where is our contract for resolving the scope. The underlying Department code can be whatever we want - a real-life example is fetching the BgovBio for a Legislator, see the BgovBio implementation here
Let's also change our code for sideloading departments at /api/v1/employees?include=departments:
class PositionResource < ApplicationResource
type :positions
- belongs_to :department,
- scope: -> { Department.all },
- foreign_key: :department_id,
- resource: DepartmentResource
+ allow_sideload :department, resource: DepartmentResource do
+ scope do |employees|
+ Department.where(employee_id: employees.map(&:id))
+ end
+
+ assign do |employees, departments|
+ employees.each do |e|
+ e.department = departments.find { |d| d.employee_id == e.id }
+ end
+ end
+ end
endclass Position < ApplicationRecord
- belongs_to :department, optional: true
+ attr_accessor :department
endclass EmployeesController < ApplicationController
jsonapi resource: EmployeeResource
def index
- render_jsonapi(Employee.all)
+ render_jsonapi(Search::Employee.new)
end
endclass EmployeeResource < ApplicationResource
type :employees
+ use_adapter JsonapiCompliable::Adapters::Trample
+ allow_filter :first_name
+ allow_filter :first_name_prefix do |scope, value|
+ scope.condition(:first_name).starts_with(value)
+ end
+ def resolve(scope)
+ scope.query!
+ scope.results
+ end
endUse the jsorm library from Node or the browser. There are also Python and Ruby clients.
This will fetch an employee with id 123. their last 3 positions where the title starts with 'dev', and the departments for those positions.
First define our models (additional client-side business logic can go in these classes):
class Employee extends Model {
static jsonapiType: 'people';
firstName: attr();
lastName: attr();
age: attr();
positions: hasMany();
}
class Position extends Model {
static jsonapiType: 'positions';
title: attr();
department: belongsTo();
}
class Department extends Model {
static jsonapiType: 'departments';
name: attr();
}Fetch the data in one call:
let positionScope = Position.where({ title_prefix: 'dev' }).order({ created_at: 'dsc' });
let scope = Employee.includes({ positions: 'department' }).merge({ positions: positionScope});
scope.find(123).then (response) => {
let employee = response.data;
// access data like so in HTML:
// employee.positions[0].department.name
}Because all of this is configuration via DSL, we can introspect that DSL and auto-generate documentation. Previously I've used swagger for this, but the UI gets awkward when many filters and relationships are present. Instead, I'd like to spend ~2 days writing something specific to this use case.
Q2/Q3 - Continue rewriting features, improving deploy pipeline (PaaS), and knowledge-sharing. Improve libraries as feedback comes in. Figure out more about single-app versus multi-app, and how we'd like to split these. Start addressing more client-side concerns (such as what framework we will use), so we have a full-stack picture.
Q4 - At this point we have shared knowledge of the concepts/DSL/API, we feel confident with the overall approach, and we've demonstrated gains to the business. At this point we can start experimenting with other languages for new features - this involves rewriting these libraries in those other languages (for Node I think this is ~1wk effort for basic use cases). We can take the "lessons learned" from Q2/Q3 and apply them here.
I find GraphQL unnecessary. If you disagree, there are two possible compromises.
Use JSONAPI libraries to resolve GraphQL: GraphQL ends up in a resolve function, which is roughly equivalent to jsonapi_scope. Use these libraries (or dopplegangers in the relevant languages) to decouple from Sequelize and make resolve easier to customize.
Support GraphQL within JSONAPI libraries: See concepts. The Query is responsible for normalizing a request. We simply have to swap out a different Query object that normalizes a GraphQL payload instead of a JSONAPI query string.
We already have Ruby/Rails experience - and years of related libraries, standards and practices. Going to the Wild West that is the NodeJS community scares the crap out of me. If it's been this hard to develop standardized code with a framework based around common conventions, it's going to be doubly hard when we remove those conventions and add a bunch of inexperienced developers to the team. So I would use Ruby and JSONAPI in order to get all the benefits to the business at the lowest cost.
That said, you may disagree with me. On the other side of this spectrum is NodeJS and GraphQL. This is still an improvement over BAS services. If we go this route, I would recommend rewriting the JSONAPI libraries in Node and calling jsonapi_scope to fulfill the resolve function. I suggest the same solution if you like the GraphQL payload but are against NodeJS.