JSON objects, schema and benchmark code

advanced_object.json

{
    fullName : "John Doe",
    age : 47,
    state : "Massachusetts",
    city : "Boston",
    zip : 02201,
    married : false,
    dozen : 12,
    dozenOrBakersDozen : 13,
    favoriteEvenNumber : 14,
    topThreeFavoriteColors : [ "red", "magenta", "cyan" ],
    favoriteSingleDigitWholeNumbers : [ 7 ],
    favoriteFiveLetterWord : "coder",
    emailAddresses : 
    [
        "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ@letters-in-local.org",
        "01234567890@numbers-in-local.net",
        "&'*+-./=?^_{}~@other-valid-characters-in-local.net",
        "mixed-1234-in-{+^}-local@sld.net",
        "a@single-character-in-local.org",
        "\"quoted\"@sld.com",
        "\"\\e\\s\\c\\a\\p\\e\\d\"@sld.com",
        "\"quoted-at-sign@sld.org\"@sld.com",
        "\"escaped\\\"quote\"@sld.com",
        "\"back\\slash\"@sld.com",
        "one-character-third-level@a.example.com",
        "single-character-in-sld@x.org",
        "local@dash-in-sld.com",
        "letters-in-sld@123.com",
        "one-letter-sld@x.org",
        "uncommon-tld@sld.museum",
        "uncommon-tld@sld.travel",
        "uncommon-tld@sld.mobi",
        "country-code-tld@sld.uk",
        "country-code-tld@sld.rw",
        "local@sld.newTLD",
        "the-total-length@of-an-entire-address.cannot-be-longer-than-two-hundred-and-fifty-four-characters.and-this-address-is-254-characters-exactly.so-it-should-be-valid.and-im-going-to-add-some-more-words-here.to-increase-the-lenght-blah-blah-blah-blah-bla.org",
        "the-character-limit@for-each-part.of-the-domain.is-sixty-three-characters.this-is-exactly-sixty-three-characters-so-it-is-valid-blah-blah.com",
        "local@sub.domains.com"
    ],
    ipAddresses : [ "127.0.0.1", "24.48.64.2", "192.168.1.1", "209.68.44.3", "2.2.2.2" ]
}

advanced_schema_v2.json

{
    name : "test",
    type : "object",
    additionalProperties : false,
    properties :
    {
        fullName                        : { type : "string" },
        age                             : { type : "integer" },
        optionalItem                    : { type : "string", optional : true },
        state                           : { type : "string", optional : true },
        city                            : { type : "string", optional : true },
        zip                             : { type : "integer", format : "postal-code" },
        married                         : { type : "boolean" },
        dozen                           : { type : "integer", minimum : 12, maximum : 12 },
        dozenOrBakersDozen              : { type : "integer", minimum : 12, maximum : 13 },
        favoriteEvenNumber              : { type : "integer", divisibleBy : 2 },
        topThreeFavoriteColors          : { type : "array", minItems : 3, maxItems : 3, uniqueItems : true, items : { type : "string", format : "color" }},
        favoriteSingleDigitWholeNumbers : { type : "array", minItems : 1, maxItems : 10, uniqueItems : true, items : { type : "integer", minimum : 0, maximum : 9 }},
        favoriteFiveLetterWord          : { type : "string", minLength : 5, maxLength : 5 },
        emailAddresses                  : { type : "array", minItems : 1, uniqueItems : true, items : { type : "string", format : "email" }},
        ipAddresses                     : { type : "array", uniqueItems : true, items : { type : "string", format : "ip-address" }},
    }
}

advanced_schema_v3.json

{
    name : "test",
    type : "object",
    additionalProperties : false,
    properties :
    {
        fullName                        : { type : "string", required : true },
        age                             : { type : "integer", required : true },
        optionalItem                    : { type : "string" },
        state                           : { type : "string" },
        city                            : { type : "string" },
        zip                             : { type : "integer", required : true, format : "postal-code" },
        married                         : { type : "boolean", required : true },
        dozen                           : { type : "integer", required : true, minimum : 12, maximum : 12 },
        dozenOrBakersDozen              : { type : "integer", required : true, minimum : 12, maximum : 13 },
        favoriteEvenNumber              : { type : "integer", required : true, divisibleBy : 2 },
        topThreeFavoriteColors          : { type : "array", required : true, minItems : 3, maxItems : 3, uniqueItems : true, items : { type : "string", format : "color" }},
        favoriteSingleDigitWholeNumbers : { type : "array", required : true, minItems : 1, maxItems : 10, uniqueItems : true, items : { type : "integer", minimum : 0, maximum : 9 }},
        favoriteFiveLetterWord          : { type : "string", required : true, minLength : 5, maxLength : 5 },
        emailAddresses                  : { type : "array", required : true, minItems : 1, uniqueItems : true, items : { type : "string", format : "email" }},
        ipAddresses                     : { type : "array", required : true, uniqueItems : true, items : { type : "string", format : "ip-address" }},
    }
}

basic_object.json

{
	fullName : "John Doe",
	age : 47,
	state : "Massachusetts",
	city : "Boston",
	zip : 02201,
	married : false,
	dozen : 12,
	dozenOrBakersDozen : 13
}

basic_schema_v2.json

{
    name : "test",
    type : "object",
    additionalProperties : false,
    properties :
    {
        fullName            : { type : "string" },
        age                 : { type : "integer" },
        optionalItem        : { type : "string", optional : true },
        state               : { type : "string", optional : true },
        city                : { type : "string", optional : true },
        zip                 : { type : "integer", format : "postal-code" },
        married             : { type : "boolean" },
        dozen               : { type : "integer", minimum : 12, maximum : 12 },
        dozenOrBakersDozen  : { type : "integer", minimum : 12, maximum : 13 }
    }
}

basic_schema_v3.json

{
    name : "test",
    type : "object",
    additionalProperties : false,
    properties :
    {
        fullName            : { type : "string", required : true },
        age                 : { type : "integer", required : true },
        optionalItem        : { type : "string" },
        state               : { type : "string" },
        city                : { type : "string" },
        zip                 : { type : "integer", required : true, format : "postal-code" },
        married             : { type : "boolean", required : true },
        dozen               : { type : "integer", required : true, minimum : 12, maximum : 12 },
        dozenOrBakersDozen  : { type : "integer", required : true, minimum : 12, maximum : 13 }
    }
}

benchmark.js

"use strict";

var test_objects = require("./test_objects"),
	test_schema = require("./test_schema"),
	jsv2 = require("JSV").JSV.createEnvironment("json-schema-draft-02"),
	jsv3 = require("JSV").JSV.createEnvironment("json-schema-draft-03"),
	schema = require("schema")(),
	jsonschemavalidate = require("json-schema"),
	assert = require("assert");

var basicSchema2 = test_schema.basic2;
var basicSchema3 = test_schema.basic3;
var basicObject = test_objects.basic;
var schemaBasicSchema2 = schema.Schema.create(test_schema.basic2);
var jsv2BasicSchema = jsv2.createSchema(basicSchema2);
var jsv3BasicSchema = jsv3.createSchema(basicSchema3);
var jsv2BasicObect = jsv2.createInstance(basicObject);
var jsv3BasicObect = jsv3.createInstance(basicObject);

assert.strictEqual(jsv2BasicSchema.validate(jsv2BasicObect).errors.length, 0);
assert.strictEqual(jsv3BasicSchema.validate(jsv3BasicObect).errors.length, 0);
assert.strictEqual(schemaBasicSchema2.validate(basicObject).isError(), false);
assert.strictEqual(jsonschemavalidate.validate(basicObject, basicSchema3).errors.length, 0);

var advancedSchema2 = test_schema.advanced2;
var advancedSchema3 = test_schema.advanced3;
var advancedObject = test_objects.advanced;
var schemaAdvancedSchema2 = schema.Schema.create(test_schema.advanced2);
var jsv2AdvancedSchema = jsv2.createSchema(advancedSchema2);
var jsv3AdvancedSchema = jsv3.createSchema(advancedSchema3);
var jsv2AdvancedObect = jsv2.createInstance(advancedObject);
var jsv3AdvancedObect = jsv3.createInstance(advancedObject);

assert.strictEqual(jsv2AdvancedSchema.validate(jsv2AdvancedObect).errors.length, 0);
assert.strictEqual(jsv3AdvancedSchema.validate(jsv3AdvancedObect).errors.length, 0);
assert.strictEqual(schemaAdvancedSchema2.validate(advancedObject).isError(), false);
assert.strictEqual(jsonschemavalidate.validate(advancedObject, advancedSchema3).errors.length, 0);


var i=0, max=10000, start, end;

console.log("BASIC:");

// JSV v2
start = Date.now();
for(i=0;i<max;i++)
{
	jsv2BasicSchema.validate(jsv2BasicObect);
}
end = Date.now();
console.log("JSV v2 total time (" + (end-start) + ") and per document time: " + ((end-start)/max));

// JSV v3
start = Date.now();
for(i=0;i<max;i++)
{
	jsv3BasicSchema.validate(jsv3BasicObect);
}
end = Date.now();
console.log("JSV v3 total time (" + (end-start) + ") and per document time: " + ((end-start)/max));

// schema
start = Date.now();
for(i=0;i<max;i++)
{
	schemaBasicSchema2.validate(basicObject);
}
end = Date.now();
console.log("schema total time (" + (end-start) + ") and per document time: " + ((end-start)/max));

// json-schema
start = Date.now();
for(i=0;i<max;i++)
{
	jsonschemavalidate.validate(basicObject, basicSchema3);
}
end = Date.now();
console.log("json-schema total time (" + (end-start) + ") and per document time: " + ((end-start)/max));


console.log("\nADVANCED:");

// JSV v2
start = Date.now();
for(i=0;i<max;i++)
{
	jsv2AdvancedSchema.validate(jsv2AdvancedObect);
}
end = Date.now();
console.log("JSV v2 total time (" + (end-start) + ") and per document time: " + ((end-start)/max));

// JSV v3
start = Date.now();
for(i=0;i<max;i++)
{
	jsv3AdvancedSchema.validate(jsv3AdvancedObect);
}
end = Date.now();
console.log("JSV v3 total time (" + (end-start) + ") and per document time: " + ((end-start)/max));

// schema
start = Date.now();
for(i=0;i<max;i++)
{
	schemaAdvancedSchema2.validate(advancedObject);
}
end = Date.now();
console.log("schema total time (" + (end-start) + ") and per document time: " + ((end-start)/max));

// json-schema
start = Date.now();
for(i=0;i<max;i++)
{
	jsonschemavalidate.validate(advancedObject, advancedSchema3);
}
end = Date.now();
console.log("json-schema total time (" + (end-start) + ") and per document time: " + ((end-start)/max));

For those who want to run this gist, you will need to install required modules:

npm install JSV schema json-schema

and create two files:

test_objects.js

eval('this.basic = ' + require('fs').readFileSync('./basic_object.json', 'utf8'));
eval('this.advanced = ' + require('fs').readFileSync('./advanced_object.json', 'utf8'));

test_schema.js

eval('this.basic2 = ' + require('fs').readFileSync('./basic_schema_v2.json', 'utf8'));
eval('this.basic3 = ' + require('fs').readFileSync('./basic_schema_v3.json', 'utf8'));
eval('this.advanced2 = ' + require('fs').readFileSync('./advanced_schema_v2.json', 'utf8'));
eval('this.advanced3 = ' + require('fs').readFileSync('./advanced_schema_v3.json', 'utf8'));

Thank you for your tests.

I added z-schema to the tests.

• z-schema supports Draft4
• json-schema supports Draft2 (unsure..)
• json-schema seems to be about double the speed of z-schema
• z-schema has 700 downloads last month
• json-schema has almost 8000 downloads

<edit: z-schema seems much faster in the advanced section. I missed a zero..>

I'm not sure, but I think json-pointers are supported in Draft4, but not in Draft2. The pointers seems very valuable in complex schemas.

I'm interested in your thoughts about draft4 vs draft2 vs speed vs community support. I don't know about any other library than z-schema supporting draft4.

BASIC:
JSV v2 total time (3576) and per document time: 0.3576
JSV v3 total time (3858) and per document time: 0.3858
schema total time (269) and per document time: 0.0269
json-schema total time (122) and per document time: 0.0122
z-schema total time (206) and per document time: 0.0206

ADVANCED:
JSV v2 total time (18118) and per document time: 1.8118
JSV v3 total time (18808) and per document time: 1.8808
schema total time (877) and per document time: 0.0877
json-schema total time (1015) and per document time: 0.1015
z-schema total time (262) and per document time: 0.0262

Oops, I misread the results.
z-schema seems to be really fast in the advanced section.

Oops again :(
I missed that z-schema is asynchronous. Not possible to do the same kind of timing as for the other libraries. How would one benchmark an asynchronous function?

Hi, I put together similar tests for v4 examples on json-schema.org page.
You can find it here: https://github.com/zaggino/z-schema/blob/master/benchmark/benchmark.js

My results:

$ node benchmark.js
tv4#basic x 26,842 ops/sec ±0.39% (98 runs sampled)
jayschema#basic x 843 ops/sec ±1.26% (93 runs sampled)
z-schema#basic x 12,995 ops/sec ±1.55% (94 runs sampled)
Fastest is tv4#basic
jayschema#advanced x 107 ops/sec ±1.39% (80 runs sampled)
z-schema#advanced x 2,294 ops/sec ±0.74% (98 runs sampled)
Fastest is z-schema#advanced