Skip to content

Instantly share code, notes, and snippets.

@joewiz
Last active June 6, 2021 00:33
Show Gist options
  • Save joewiz/d986da715facaad633db to your computer and use it in GitHub Desktop.
Save joewiz/d986da715facaad633db to your computer and use it in GitHub Desktop.
An implementation of XQuery 3.1's fn:json-to-xml and fn:xml-to-json functions for eXist
xquery version "3.1";
(:~
: An implementation of XQuery 3.1's fn:json-to-xml and fn:xml-to-json functions for eXist, which does not support them natively as of 4.3.0.
:
: @author Joe Wicentowski
: @version 0.4
: @see http://www.w3.org/TR/xpath-functions-31/#json
:)
module namespace jx = "http://joewiz.org/ns/xquery/json-xml";
(:~
: Parses a string supplied in the form of a JSON text, returning the results in the form of an XML document node.
:
: @param $json-text A string supplied in the form of a JSON text
: @return The results in the form of an XML document node
: @see https://www.w3.org/TR/xpath-functions-31/#func-json-to-xml
:)
declare function jx:json-to-xml($json-text as xs:string) as document-node()? {
jx:json-to-xml($json-text, map {})
};
(:~
: Parses a string supplied in the form of a JSON text, returning the results in the form of an XML document node.
:
: @param $json-text A string supplied in the form of a JSON text
: @param $options Used to control the way in which the parsing takes place
: @return The results in the form of an XML document node
: @see https://www.w3.org/TR/xpath-functions-31/#func-json-to-xml
:)
declare function jx:json-to-xml($json-text as xs:string, $options as map(*)) as document-node()? {
let $json := parse-json($json-text, $options)
return
document { jx:json-to-xml-recurse($json) }
};
(:~
: Converts an XML tree, whose format corresponds to the XML representation of JSON defined in the XPath and XQuery 3.1 Functions & Operators specification, into a string conforming to the JSON grammar.
:
: @param $input An XML tree, whose format corresponds to the XML representation of JSON
: @return A string conforming to the JSON grammar
: @see https://www.w3.org/TR/xpath-functions-31/#func-xml-to-json
:)
declare function jx:xml-to-json($input as node()?) as xs:string? {
jx:xml-to-json($input, map {} )
};
(:~
: Converts an XML tree, whose format corresponds to the XML representation of JSON defined in the XPath and XQuery 3.1 Functions & Operators specification, into a string conforming to the JSON grammar.
:
: @param $input An XML tree, whose format corresponds to the XML representation of JSON
: @param $options Options for controlling the way in which the conversion takes place
: @return A string conforming to the JSON grammar
: @see https://www.w3.org/TR/xpath-functions-31/#func-xml-to-json
:)
declare function jx:xml-to-json($input as node()?, $options as map(*)) as xs:string? {
let $json := jx:xml-to-json-recurse($input)
let $serialization-parameters := map { "method": "json", "indent": $options?indent }
return
serialize($json, $serialization-parameters)
};
(:~
: A utility function that recurses through a parsed JSON text, returning the results in the form of XML nodes.
:
: @param $json A parsed JSON text
: @return The results in the form of an XML document node
:)
declare %private function jx:json-to-xml-recurse($json as item()*) as item()+ {
let $data-type := jx:json-data-type($json)
return
element { QName("http://www.w3.org/2005/xpath-functions", $data-type) } {
if ($data-type eq "array") then
for $array-member in $json?*
let $array-member-data-type := jx:json-data-type($array-member)
return
element {$array-member-data-type} {
if ($array-member-data-type = ("array", "map")) then
jx:json-to-xml-recurse($array-member)/node()
else
$array-member
}
else if ($data-type eq "map") then
map:for-each(
$json,
function($object-name, $object-value) {
let $object-value-data-type := jx:json-data-type($object-value)
return
element { QName("http://www.w3.org/2005/xpath-functions", $object-value-data-type) } {
attribute key {$object-name},
if ($object-value-data-type = ("array", "map")) then
jx:json-to-xml-recurse($object-value)/node()
else
$object-value
}
}
)
else
$json
}
};
(:~
: A utility function for getting the data type of JSON data
:)
declare %private function jx:json-data-type($json as item()?) {
if ($json instance of array(*)) then 'array'
else if ($json instance of map(*)) then 'map'
else if ($json instance of xs:string) then 'string'
else if ($json instance of xs:double) then 'number'
else if ($json instance of xs:boolean) then 'boolean'
else if (empty($json)) then 'null'
else error(xs:QName('ERR'), 'Not a known data type for json data')
};
declare %private function jx:xml-to-json-recurse($input as node()*) as item()* {
for $node in $input
return
typeswitch ($node)
case element(fn:map) return
if ($node/@key) then
map { $node/@key: map:merge( jx:xml-to-json-recurse($node/node()) ) }
else
map:merge( jx:xml-to-json-recurse($node/node()) )
case element(fn:array) return
if ($node/@key) then
map { $node/@key: array { jx:xml-to-json-recurse($node/node()) } }
else
array { jx:xml-to-json-recurse($node/node()) }
case element(fn:string) return
if ($node/@key) then
map { $node/@key: $node/string() }
else
$node/string()
case element(fn:number) return
if ($node/@key) then
map { $node/@key: $node cast as xs:double }
else
$node cast as xs:double
case element(fn:boolean) return
if ($node/@key) then
map { $node/@key: $node cast as xs:boolean }
else
$node cast as xs:boolean
case element(fn:null) return
if ($node/@key) then
map { $node/@key: () }
else
'null'
case document-node() return
jx:xml-to-json-recurse($node/node())
(: Comments, processing instructions, and whitespace text node children of map and array are ignored :)
case text() return
if (normalize-space($node) eq '') then
()
else
$node
case comment() | processing-instruction() return
()
case element() return
error(xs:QName('FOJS0006'), 'Invalid XML representation of JSON')
default return
error(xs:QName('ERR'), 'Does not match known node types for xml-to-json data')
};
@joewiz
Copy link
Author

joewiz commented Oct 5, 2015

I've recently fixed some issues here:

  • Fixed errors involving map and array types on lines 12-13
  • Fixed namespace issues on line 42

@joewiz
Copy link
Author

joewiz commented Oct 5, 2015

Added ju:xml-to-json()

@joewiz
Copy link
Author

joewiz commented Aug 6, 2017

  • Updated jx:json-to-xml() and jx:xml-to-json() to conform to final XPath and XQuery 3.1 Functions & Operators spec, with changes affecting function signatures, options parameters, namespaces of returned elements.
  • Added full xqdoc comments
  • Changed module filename from json-util to json-xml, and namespace from ju = "http://joewiz.org/ns/xquery/json-util" to jx = "http://joewiz.org/ns/xquery/json-xml"

@dizzzz
Copy link

dizzzz commented Aug 7, 2017

interesting stuff..... any test code available?

@joewiz
Copy link
Author

joewiz commented Mar 30, 2018

Fixed error reported on exist-open.

@a-tab
Copy link

a-tab commented Jul 19, 2018

Thank you for this code! I've been incorporating it into my system as a replacement for XQJSON and it has been working very well, but at one point I was attempting to convert an XML structure with nested map objects for Amazon AWS's Elastic Transcoder. When the inner map contained a @key value, it was turned into an array in the output. Example:
let $metadata-string := jx:xml-to-json( <fn:map> <fn:string key="Name">pipeline-name</fn:string> <fn:string key="InputBucket">input-bucket</fn:string> <fn:string key="OutputBucket">output-bucket</fn:string> <fn:string key="Role">pipeline-role</fn:string> <fn:map key="Notifications"> <fn:string key="Progressing"/> <fn:string key="Completed">transcoder-complete</fn:string> <fn:string key="Warning"/> <fn:string key="Error"/> </fn:map> </fn:map>)

I believe this has to do with line 128. When I changed it to the following, my issue was resolved.
map { $node/@key: map:merge(jx:xml-to-json-recurse($node/node())) }

@joewiz
Copy link
Author

joewiz commented Jul 21, 2018

@a-taboada You're quite right. I've updated the module with your fix, and made other minor fixes after running the XQuery Test Suite's tests for fn:xml-to-json against this code. I'll keep working to improve the performance of the module against XQTS. Thanks for the report!

@adamretter
Copy link

@joewiz there is no map:for-each-entry it should be map:for-each... especially for eXist-db 5.0.0 ;-)

Do you plan to release this as a XAR?

@joewiz
Copy link
Author

joewiz commented May 29, 2019

@adamretter I've fixed the issue now. Thank you! I only just noticed your comment, since gist doesn't issue notifications. (Please always email/dm me!)

@joewiz
Copy link
Author

joewiz commented May 29, 2019

OMG, I just got a notification with my response! Did they actually enable notifications on gist!? Yes, yes, they did! See isaacs/github#21 (comment) and https://help.github.com/en/articles/about-gists#receiving-notifications-for-gist-activity.

@adamretter
Copy link

Thanks Joe

@benibela
Copy link

benibela commented Sep 11, 2020

This does not check for the "escape" option? Do you plan to add it?

@joewiz
Copy link
Author

joewiz commented Sep 11, 2020

@benibela Now that eXist has native support for fn:json-to-xml() and fn:xml-to-json(), I don't use this library anymore. You're welcome to fork this and add to it though. If you do, please add a link here, and I bet people would find it useful!

@benibela
Copy link

I have changed it a little to pass more qt3 tests https://gist.github.com/benibela/0bd905319eb159353f0620ffb3ab65a5

Unfortunately xml-to-json/json-to-xml need to do things that parse-json/serialize cannot do, like preserving nulls. So this is kind of a deadend

@joewiz
Copy link
Author

joewiz commented Sep 24, 2020

@benibela Thanks for the link to your fork. Could you give an example of how this approach fail to preserve nulls?

@benibela
Copy link

benibela commented Sep 24, 2020

Because null is parsed as empty-sequence, and then it disappears in the iteration, [null, null] turn into []

Although on second thought, it can actually work with null, but it needs to iterate differently

I have updated it. Only failing 56 of 168 tests now

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment