- What/Why JSON schema
- Apply to rails model validation
- Test your API endpoint with schema matcher
- Homework for a curious reader
- References
What is JSON schema?
I recommend you to read this and that to understand more about JSON schema.
However if you are lazy like me, just think JSON schema as the spec of your JSON data, it help you defines how your JSON data should looks like. The simplest schema is like below:
{
"properties": {
"name": {
"type": "string"
}
},
"required": ["name"],
"type": "object"
}This schema means your JSON object should have a name attribute with string type.
for example, this will be a valid JSON:
{
"name": "Wayne"
}However this is not a valid JSON because name is a number but not a string
{
"name": 5566
}So, why we need JSON schema? What's the benefit?
First of all, define your data properly is never a bad idea. And there are at least 4 benefits I can think of:
- validate your JSON data structure, so you don't mess up your database
- help you validate your APIs response, especially REST-like API
- One rule, everywhere. Help your client validate their data.
- Bonus: can integrate with Swagger (if you like)
Cool, so now let's write some code with our beloved ruby.
There are 2 gems I found to help me validate JSON Schema, the first one is the ruby implementation of JSON Schema validate called json-schema, the second one is rails validator implementation called activerecord_json_validator based on first one.
Let's use activerecord_json_validator to integrate our model level JSON validation.
Assume we have User and Report, we allow user to send error report to us including their system's environment and save at data column as JSON, migration file looks like below:
# db/migrations/xxxxxxxxx_create_reports.rb
class CreateReports < ActiveRecord::Migration
def change
create_table :reports do |t|
t.references :user
t.jsonb :data, null: false, default: "{}"
t.timestamps null: false
end
end
endWe want our Report#data to have at least 2 keys devise_id and version, so a valid JSON should be like below:
{
"devise_id": "devise-id-is-a-string",
"version": "5.56.6"
}To test our model validation, we write rspec code like below:
# spec/models/report_spec.rb
RSpec.describe Report, type: :model do
describe 'validates data column' do
# We use Factory girl to create fake record
subject(:report) { create(:report, data: data) }
let(:valid_data) do
{
devise_id: 'devise-id-is-a-string',
version: '5.56.6'
}
end
describe 'valid data' do
let(:data) { valid_data }
it 'creates report' do
expect { report }.to change { Report.count }.by(1)
end
end
describe 'invalid data' do
context 'when missing devise_id' do
let(:data) { valid_data.except(:devise_id) }
it 'raise validation error' do
expect { report }.to raise_error(ActiveRecord::RecordInvalid)
end
end
context 'when missing version' do
let(:data) { valid_data.except(:version) }
it 'raise validation error' do
expect { report }.to raise_error(ActiveRecord::RecordInvalid)
end
end
end
end
endInstall gem
# Gemfile
gem 'activerecord_json_validator'Add validation into Report
# app/models/report.rb
class Report < ActiveRecord::Base
JSON_SCHEMA = "#{Rails.root}/app/models/schemas/report/data.json"
belongs_to :user
validates :data, presence: true, json: { schema: JSON_SCHEMA }
endAdd JSON schema file
I prefer add .json file into app/models/schemas/report/data.json
// app/models/schemas/report/data.json
{
"$schema": "http://yourdomain.com/somewhere/report/data",
"type": "object",
"properties": {
"devise_id": {
"type": "string"
},
"version": {
"type": "string"
}
},
"required": [
"devise_id",
"version"
]
}Now all test passed.
Now it's time to add some api endpoint response test.
Assume we have users api GET /users and GET /users/:id, lets define our response:
// GET /users
{
"users": [
{
"id": 1,
"name": "John John Slater",
"email": "jjs@example.com",
"is_good_surfer": true,
"updated_at": "timestamp",
"created_at": "timestamp"
}
]
}
// GET /users/:id
{
"user": {
"id": 1,
"name": "John John Slater",
"email": "jjs@example.com",
"is_good_surfer": true,
"updated_at": "2017-02-01T10:00:54.326+10:00",
"created_at": "2017-02-01T10:00:54.326+10:00"
}
}Lets write some tests first
I'm using Rspec and found out there is a gem called json_matcher, it's also based on gem json-schema, so you can choose either one to implement your test, check this post and you will have more idea how to do this.
Ok, time to make our hand dirty.
First do some setup:
# Gemfile
gem 'json_matchers'
# spec/spec_helper.rb
require "json_matchers/rspec"
# spec/support/json_matchers.rb
JsonMatchers.schema_root = "controller/schemas"And write tests
RSpec.describe User, type: :request do
describe 'GET /users' do
let!(:user) { create(:user) }
subject! { get '/users' }
specify do
expect(response).to be_success
expect(response).to match_response_schema('users')
end
end
describe 'GET /users/:id' do
let(:user) { create(:user) }
subject! { get "/users/#{user.id}" }
specify do
expect(response).to be_success
expect(response).to match_response_schema('user')
end
end
endTest should be failed because we haven't added JSON schema file yet.
Add schema file for user, notice we put user object definitions into definitions so we can reuse when we define users.json
// app/controllers/schemas/user.json
{
"type": "object",
"required": ["user"],
"properties": {
"user": {
"$ref": "#/definitions/user"
}
},
"definitions": {
"user": {
"type": "object",
"required": [
"id",
"name",
"email",
"is_good_surfer",
"updated_at",
"created_at"
],
"properties": {
"id": {
"type": "integer"
},
"name": {
"type": "string"
},
"email": {
"type": "string"
},
"is_good_surfer": {
"type": "boolean"
},
"updated_at": {
"type": "string"
},
"created_at": {
"type": "string"
}
}
}
}
}// app/controllers/chemas/users.json
{
"type": "object",
"required": ["users"],
"properties": {
"users": {
"items": {
"$ref": "user.json#/definitions/user"
},
"type": "array"
}
}
}In here we've used a technic called reference, as you can see we put user object's schema into definitions inside app/controllers/schemas/user.json and we use "$ref": "#/definitions/user" to reference it, it's a best practice to move your schema under definition key so that you can reuse it in the future (even cross file), as you see we also use "$ref": "user.json#/definitions/user" inside our app/controllers/chemas/users.json, just think it as rails view partial and you'll get it. 😎
Let's run our test, now test passed.
As so once again, the day is saved thanks to JSON schema. 😂
Hope you had a good reading.
If this post can't fulfill your curiosity, there are more topics of JSON schema you can chasing for:
- How to write generic JSON schema test and apply to every api endpoints?
- How to expose your JSON schema so you can share/use it either at rails project or other repos
Also, you can check those reference for more details. Cheers!
- https://brandur.org/elegant-apis
- https://robots.thoughtbot.com/validating-json-schemas-with-an-rspec-matcher
- https://github.com/mirego/activerecord_json_validator
- https://github.com/ruby-json-schema/json-schema
- https://robots.thoughtbot.com/validating-the-formkeep-api
- https://spacetelescope.github.io/understanding-json-schema/
- http://jsonschema.net/#/
@wayne5540 looking good! The two bits that I would probably expand on at this point:
$ref- I think this is an important and often-missed trick that allows one to DRY their schemas, so I reckon it warrants a bit more time and focus. E.g. say why you're doing this, and maybe give an example of how it can be reused in another schema.Finally, "further reading" part - do you intend it to be topics of future blogs posts? Or "homework for a curious reader" kind of thing? I think the title is slightly misleading, so trying to understand what you're going for here before suggesting an alternative.