Skip to content

Instantly share code, notes, and snippets.

@ahx
Last active June 24, 2024 09:08
Show Gist options
  • Save ahx/1538c31f0652f459861713b5259e366a to your computer and use it in GitHub Desktop.
Save ahx/1538c31f0652f459861713b5259e366a to your computer and use it in GitHub Desktop.
Comparing openapi_first and committee

This is a feature comparison of openapi_first and committee

I will try to keep this updated. Please create an issue if you see something wrong/missing here.

JSON Schema validation

Most of these differences are because openapi_first uses json_schemer, which has build in support for OpenAPI 3.1 where committee/openapi_parser currently ships it's own implementation.

Feature openapi_first committee
Support OpenAPI 3.1 (Published 15 February 2021) issue
Validate enums issue
Support explode: true in array query parameters
Support if/else in JSON Schemas
Support writeOnly/readOnly ✅ (docs) issue
Support all format validations (like format: date) issue
Support nullable: true (removed in OpenAPI 3.1)
Support type arrays (type: [string, 'null']) (OpenAPI 3.1)
Support discriminator

Parsing parameters

Feature openapi_first committee
Parse and validate query parameters that include brackets (?names[]=1")
Parse parameters exactly as described. Support all explode and style options. issue issue
Convert simple parameter values (integer, number)
Support named nested parameters (name: 'filter[thing]') issue

Other

Feature openapi_first committee
$ref just works issue issue issue
Validate request body
Validate request parameters
Validate response body
Validate response headers ⚠️ committee does not complain about missing required response headers issue
Coverage
Test assertions
Option to raise exception if request/response is invalid
Option to raise exception if request is not specified
Complain about unkown response body content type ⚠️ (committee ignores unknown response bodies by default unless you use the strict option)
Convert date, date-time request values to Date, DateTime
Support custom error response
Add parsed parameters, body to rack env
Add OpenAPI request object to rack env to enable runtime inspection
Middleware has configurable callback when validation error occurs ☑️ (use hooks)

Error messages during development

Error: Referenced file could not be found

committee:

NoMethodError:
    undefined method `set_path_item_to_operation' for an instance of OpenAPIParser::Schemas::OpenAPI
  # ./spec/support/web_helpers.rb:6:in `app'

or

  OpenAPIParser::MissingReferenceError:
    './paths/access-control_roles.yaml#/components/paths/access-control_roles' was referenced but could not be found
  # ./spec/support/web_helpers.rb:6:in `app'

openapi_first:

  OpenapiFirst::FileNotFoundError:
    Problem while loading file referenced in openapi/openapi.yaml: File not found /Users/ahaller/src/access-control-api/openapi/schemas/PermissionObject.yaml
  # /Users/ahaller/src/openapi_first/lib/openapi_first/json_refs.rb:144:in `load_referenced_file'
  # /Users/ahaller/src/openapi_first/lib/openapi_first/json_refs.rb:137:in `dereference_file'
  # /Users/ahaller/src/openapi_first/lib/openapi_first/json_refs.rb:122:in `referenced_value'
...

JSON Schema validation error: unknown properties when using additionalProperties: false

committee:

Committee::InvalidResponse:
  #/components/schemas/RoleInIndex does not define properties: type, attributes, relationships

openapi_first.

OpenapiFirst::ResponseInvalidError:
  Response body is invalid: object property at `/data/0/type` is not defined and schema does not allow additional properties. object property at `/data/0/attributes` is not defined and schema does not allow additional properties. object property at `/data/0/relationships` is not defined and schema does not allow additional properties. object at `/data/0` is missing required properties: name, nameShort, description, iwfmUserGroupUrl, createdAt, updatedAt, isDeletable, isReadonly, isAdmin, forceOnAllUsers, category. ... (16 errors total)
@andrew-platonov
Copy link

Have you compared performance issues (latancy and troughput)?

@ahx
Copy link
Author

ahx commented Sep 1, 2022

Hi @andrew-platonov there are some benchmarks here

I just added a benchmark using wrk. Here is the output comparing committee and openapi_first. Note that both are using response validation, which I usually would not enable in production:

Results

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment