Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
blog post

Validate JSON schema in Rails

Topics

  1. What/Why JSON schema
  2. Apply to rails model validation
  3. Test your API endpoint with schema matcher
  4. Homework for a curious reader
  5. References

What/Why JSON schema

What is JSON schema?

I recommend you to read this and that to understand more about JSON schema.

However if you are lazy like me, just think JSON schema as the spec of your JSON data, it help you defines how your JSON data should looks like. The simplest schema is like below:

{
  "properties": {
    "name": {
      "type": "string"
    }
  },
  "required": ["name"],
  "type": "object"
}

This schema means your JSON object should have a name attribute with string type.

for example, this will be a valid JSON:

{
  "name": "Wayne"
}

However this is not a valid JSON because name is a number but not a string

{
  "name": 5566
}

So, why we need JSON schema? What's the benefit?

First of all, define your data properly is never a bad idea. And there are at least 4 benefits I can think of:

  • validate your JSON data structure, so you don't mess up your database
  • help you validate your APIs response, especially REST-like API
  • One rule, everywhere. Help your client validate their data.
  • Bonus: can integrate with Swagger (if you like)

Cool, so now let's write some code with our beloved ruby.

Apply to rails model validation

There are 2 gems I found to help me validate JSON Schema, the first one is the ruby implementation of JSON Schema validate called json-schema, the second one is rails validator implementation called activerecord_json_validator based on first one.

Let's use activerecord_json_validator to integrate our model level JSON validation.

Assume we have User and Report, we allow user to send error report to us including their system's environment and save at data column as JSON, migration file looks like below:

# db/migrations/xxxxxxxxx_create_reports.rb
class CreateReports < ActiveRecord::Migration
  def change
    create_table :reports do |t|
      t.references :user
      t.jsonb :data, null: false, default: "{}"
      t.timestamps null: false
    end
  end
end

We want our Report#data to have at least 2 keys devise_id and version, so a valid JSON should be like below:

{
  "devise_id": "devise-id-is-a-string",
  "version": "5.56.6"
}

To test our model validation, we write rspec code like below:

# spec/models/report_spec.rb
RSpec.describe Report, type: :model do
  describe 'validates data column' do
    # We use Factory girl to create fake record
    subject(:report) { create(:report, data: data) }
    let(:valid_data) do
      {
        devise_id: 'devise-id-is-a-string',
        version: '5.56.6'
      }
    end

    describe 'valid data' do
      let(:data) { valid_data }
      it 'creates report' do
        expect { report }.to change { Report.count }.by(1)
      end
    end

    describe 'invalid data' do
      context 'when missing devise_id' do
        let(:data) { valid_data.except(:devise_id) }

        it 'raise validation error' do
          expect { report }.to raise_error(ActiveRecord::RecordInvalid)
        end
      end

      context 'when missing version' do
        let(:data) { valid_data.except(:version) }

        it 'raise validation error' do
          expect { report }.to raise_error(ActiveRecord::RecordInvalid)
        end
      end
    end
  end
end

Install gem

# Gemfile
gem 'activerecord_json_validator'

Add validation into Report

# app/models/report.rb
class Report < ActiveRecord::Base
  JSON_SCHEMA = "#{Rails.root}/app/models/schemas/report/data.json"

  belongs_to :user

  validates :data, presence: true, json: { schema: JSON_SCHEMA }
end

Add JSON schema file

I prefer add .json file into app/models/schemas/report/data.json

// app/models/schemas/report/data.json
{
  "$schema": "http://yourdomain.com/somewhere/report/data",
  "type": "object",
  "properties": {
    "devise_id": {
      "type": "string"
    },
    "version": {
      "type": "string"
    }
  },
  "required": [
    "devise_id",
    "version"
  ]
}

Now all test passed.

Test your API endpoints with schema matcher

Now it's time to add some api endpoint response test.

Assume we have users api GET /users and GET /users/:id, lets define our response:

// GET /users
{
  "users": [
    {
      "id": 1,
      "name": "John John Slater",
      "email": "jjs@example.com",
      "is_good_surfer": true,
      "updated_at": "timestamp",
      "created_at": "timestamp"
    }
  ]
}

// GET /users/:id
{
  "user": {
    "id": 1,
    "name": "John John Slater",
    "email": "jjs@example.com",
    "is_good_surfer": true,
    "updated_at": "2017-02-01T10:00:54.326+10:00",
    "created_at": "2017-02-01T10:00:54.326+10:00"
  }
}

Lets write some tests first

I'm using Rspec and found out there is a gem called json_matcher, it's also based on gem json-schema, so you can choose either one to implement your test, check this post and you will have more idea how to do this.

Ok, time to make our hand dirty.

First do some setup:

# Gemfile
gem 'json_matchers'

# spec/spec_helper.rb
require "json_matchers/rspec"

# spec/support/json_matchers.rb
JsonMatchers.schema_root = "controller/schemas"

And write tests

RSpec.describe User, type: :request do
  describe 'GET /users' do
    let!(:user) { create(:user) }
    subject! { get '/users' }

    specify do
      expect(response).to be_success
      expect(response).to match_response_schema('users')
    end
  end

  describe 'GET /users/:id' do
    let(:user) { create(:user) }
    subject! { get "/users/#{user.id}" }

    specify do
      expect(response).to be_success
      expect(response).to match_response_schema('user')
    end
  end
end

Test should be failed because we haven't added JSON schema file yet.

Add schema file for user, notice we put user object definitions into definitions so we can reuse when we define users.json

// app/controllers/schemas/user.json
{
  "type": "object",
  "required": ["user"],
  "properties": {
    "user": {
      "$ref": "#/definitions/user"
    }
  },
  "definitions": {
    "user": {
      "type": "object",
      "required": [
        "id",
        "name",
        "email",
        "is_good_surfer",
        "updated_at",
        "created_at"
      ],
      "properties": {
        "id": {
          "type": "integer"
        },
        "name": {
          "type": "string"
        },
        "email": {
          "type": "string"
        },
        "is_good_surfer": {
          "type": "boolean"
        },
        "updated_at": {
          "type": "string"
        },
        "created_at": {
          "type": "string"
        }
      }
    }
  }
}
// app/controllers/chemas/users.json
{
  "type": "object",
  "required": ["users"],
  "properties": {
    "users": {
      "items": {
        "$ref": "user.json#/definitions/user"
      },
      "type": "array"
    }
  }
}

In here we've used a technic called reference, as you can see we put user object's schema into definitions inside app/controllers/schemas/user.json and we use "$ref": "#/definitions/user" to reference it, it's a best practice to move your schema under definition key so that you can reuse it in the future (even cross file), as you see we also use "$ref": "user.json#/definitions/user" inside our app/controllers/chemas/users.json, just think it as rails view partial and you'll get it. 😎

Let's run our test, now test passed.

As so once again, the day is saved thanks to JSON schema. 😂

Hope you had a good reading.

Homework for a curious reader

If this post can't fulfill your curiosity, there are more topics of JSON schema you can chasing for:

  • How to write generic JSON schema test and apply to every api endpoints?
  • How to expose your JSON schema so you can share/use it either at rails project or other repos

References

Also, you can check those reference for more details. Cheers!

@morhekil

This comment has been minimized.

Copy link

commented Nov 20, 2016

@wayne5540 looking good! The two bits that I would probably expand on at this point:

  1. Introduction - just a quick one-two paragraphs writeup on why/what, rather that merely pointers to other articles only, to start with some context.
  2. When referencing user definition via $ref - I think this is an important and often-missed trick that allows one to DRY their schemas, so I reckon it warrants a bit more time and focus. E.g. say why you're doing this, and maybe give an example of how it can be reused in another schema.

Finally, "further reading" part - do you intend it to be topics of future blogs posts? Or "homework for a curious reader" kind of thing? I think the title is slightly misleading, so trying to understand what you're going for here before suggesting an alternative.

@morhekil

This comment has been minimized.

Copy link

commented Nov 22, 2016

@wayne5540 I revised the reference paragraph a bit, have a look here: https://gist.github.com/morhekil/3f904116f4fc120a1c3c5db4163b37b5

Looks good to me otherwise!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.