Note
This draft is elaborated based on the talk between Jachym Cepicky and George Percivall (CTO at OGC) at ISPRS congress, Prague 2016. Among other topics, we hit the JSON data format at OGCs position to it. This draft was initially written by Jachym Cepicky - who has no relevant IT education and his background is self-learned GIS and programming. Jachym is not member of OGC, therefore his statements are statements of an external observer and OGC standards consumer.
JSON (JavaScript Object Notation) file format [1] became to be very popular among software developers. While it started to be used extensively in the Web development (since derived from JavaScript programming language), it was adopted by many other programming platforms.
JSON seems to be the format, what people are considering, when trying to replace XML. There might be other formats around, which are not so popular (yet), the discussion nowadays is always spinning around JSON vs. XML formats (with one small exception: binary formats) -- see list of open data formats at Wikipedia [2].
Some of the reasons, why JSON became so popular could be (especially when compared with XML):
- Readability
- Humans can still read it
- Simplicity
- At first sight (with eyes of a human), the format is very "simple", the structure is very straight forward, derived from most currently used programming languages (at least from the "C" development branch) the structure is described probably in all programming books - everybody is familiar with it.
- Simple and "cheap" parsing
- The parsing techniques are relatively "cheap", especially compared with XML. This is significant especially, when limited resources are at hand, such as in web environment.
- Compactness
- JSON is very compact, therefore usually smaller file size - but the compactness does not affect it's great readability (from humans point of view).
- Support for NULL value
- You simply write null
When discussing advantages of XML format, 2 key features are usually named:
- Namespaces
- Namespaces do allow for sharing of standard structures, they are part of the schema system
- Schemas and data validation
- Data validation is very important part of each specification - they can ensure, that both systems (server-client) are operating within the same domain, with same data structures
Especially schema validation is very important feature of any standard (therefore, it's important for OGC too). XML schema validation is very powerful technology, and it's used in all OGC standards. Thanks to namespaces, the schemas of various standards can be linked together, therefore one standard can re-use other standard's features (e.g. from OWS Common spec.)
JSON might be looking as simple format, but still there are possibilities, how to validate JSON data structures.
JSON-Schema [3] is promising tool, which might be used for OGC-JSON encoded specs. to be evaluated against. The schemas are again JSON-encoded files with description of inner structure and data types of JSON-to-be evaluated. It seems to be simple to write and there are already tools available in Python or JavaScript programming languages, which are supporting JSON-Schema validation.
As already mentioned, namespaces are important, for linking various documents together and to make sure, the systems are communicating in the same domain level. On the other hand, namespaces are making things complicated (from the humans point of view). Also when namespaces are introduced to JSON (which is possible), things start to be chaotic at once. One of possible approach is JSON-LD project [4], also to be found as W3C recommendation [5], where the "namespace" topic is somehow hit. I confess, this topic needs wider research.
Within Geospatial industry, JSON format was already widely accepted, especially as data exchange format between server-client (usually proprietary APIs) and as GeoJSON format. People (developers, users) are already asking, whether we could adopt JSON as data-exchange format for OGC standards. Some sample implementations are already in place.
It is already adopted by some OGC OWS implementations, such as GeoServer [13] or it is used in REST APIs of various tools.
- GeoJSON
GeoJSON format [6] was already adopted by The Internet Engineering Task Force (IETF) as "standard" [7] (among others, Sean Gillies [8] was involved, who is recognised as one of the key persons in current (Python) geospatial industry). GeoJSON format enables us to encode geospatial features with their attributes to JSON structure. There are already schemas available, for the data evaluation, but they are one-man attempts and so far no GeoJSON-schema was widely spread.
Still GeoJSON started to be used widely especially in the web-development field, but not only.
- TopoJSON
Another interesting JSON-based data format is the work of Mike Bostock, who is (especially) JavaScript visionary, who is pushing (not only) the geospatial data to the web -- the TopoJSON data format [9]. TopoJSON format introduces first topological way of storing geospatial features, which is efficient especially for the polygon data, as well as storing relative coordinates as numbers of pixels from the beginning of the feature (depending on final image resolution) instead of full coordinate numbers - this leads to massive data compression. The loss is of course data precision (depending on the resolution) and some of the data types can not be stored in TopoJSON format (Big Integers, which are used as feature identifier a lot).
Based on my small research [10], when converting the input data (several thousands of polygons from OpenStreetMap dataset) with original size to TopoJSON format (from original GML), the resulting file size was about 5% of the original file (80MB to 3.3 MB - 0.5kB, if compressed)! TopoJSON is certainly not the best format for every geospatial use cases, it just shows potential of the JSON format.
- Vector Tiles
- Both GeoJSON and TopoJSON are sometimes used in various vector tiles specification. The services are using on similar principles, as raster tile serverices (TMS, WMTS) are working and they are implemented in major (open source) JavaScript mapping frameworks (Leaflet and OpenLayers).
JSONIX [11] is JavaScript library, provided by Alexey Valikov [12] it is a mapping library, which means, it provides data format conversion between XML and JSON formats based in input (pre-defined) mapping file. Alexey already provides within his repository mapping files for OGC standards - they are based on XML schemas, provided by OGC.
With help of JSONIX, we are able to transform OGC XML files to JSON format and back, with all necessary data attributes for their validation (e.g. making sure, that tag content might be number and not string etc.). Thought JSONIX is written in JavaScript (and the build system seems to be in Java used Maven), it's possible to deploy it to Web browser as well as to use it in server-side JavaScript scripts.
JSONIX shows possible way to OGC standards, how they could fit into JSON data format relatively easily. Unluckily, it is not technologically independent, since written in JavaScript, but re-implementation in other tools should be possible. Alexey gave nice presentation to this topic at FOSS4G-2015, South Korea [14].
JSON is from human point of view simple format, therefore OGC, while adopting JSON to it's structure, should make efforts to KISS. The main ideas, coming to my mind regarding OGC and JSON are
- Accept GeoJSON as standard (even adopted by another, maybe competing organisation)
- Try to find a official way, how to convert current XML documents to JSON (maybe based on JSONIX, or find even more simpler way of doing it)
- KISS - where appropriate, better lower our sights from hard coding everything, in favour of simplicity - not only machines are interacting with our standards.
- Provide JSON-Schema, where appropriate, so people could evaluate
As stated several times, JSON is popular for it's simplicity and efficiency. GeoJSON adds some structure to JSON, which already opens questions and discussions (Shall the coordinates be WGS84-only or shall we enable other CRSs too? And how?) still, GeoJSON format is quite popular and simple. If OGC starts to add complexity without very well discussed reasoning, it might fail by the community. We are looking for more easy way how to exchange messages between the server and the client, not another format, with same level of complexity, as XML documents have now.
| [1] | http://json.org JSON data format |
| [2] | https://en.wikipedia.org/wiki/List_of_file_formats Wikipedia list of formats |
| [3] | http://json-schema.org/ JSON-Schema validation |
| [4] | http://json-ld.org/ JSON Linked data |
| [5] | https://www.w3.org/TR/json-ld/ JSON Linked data - W3C Recommendation |
| [6] | http://geojson.org/ GeoJSON data format |
| [7] | https://datatracker.ietf.org/wg/geojson/ GeoJSON IETF spec. |
| [8] | https://sgillies.net/ Sean Gilies home page |
| [9] | https://github.com/mbostock/topojson TopoJSON data format from Mike Bostock |
| [10] | http://www.slideshare.net/jachym/cepicky-osmappingnotes Presentation Open Source JavaScript Mapping Frameworks page 77. |
| [11] | https://github.com/highsource/jsonix Powerful XML<->JSON JavaScript mapping library. |
| [12] | https://github.com/highsource Alexey Valikov |
| [13] | http://docs.geoserver.org/latest/en/user/services/wms/reference.html#exceptions JSON used in GeoServer |
| [14] | https://www.meci.co.kr/societyevent/FOSS4G2015/download/presentation/GBR%20B/3.%20S2-1/PT2-02_Jsonix%20Talking%20to%20OGC%20Web%20Services%20in%20JSON.pdf JSONIX FOSS4G talk |
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
