View environment-variables.xq
xquery version "3.1";
element environment-variables {
for $var in available-environment-variables()
order by $var
element environment-variable {
attribute name { $var },
attribute value { environment-variable($var) }

Web Scraping with XQuery


Learning how to web scrape empowers you to apply your XQuery skills to any data residing on the web. You can fetch data from remote sites and services—for example, entire web pages or just the pieces of a page that matter to you. Once fetched, you can perform further analysis on the data, clean it up, mash it up with other data, transform it into different formats, etc.

Built-in functions for making HTTP requests

XPath-based languages like XQuery offer an standard function for accessing remote documents, the fn:doc() function. However, a limitation of this function is that it only works if the URI returns a well-formed XML document.

cd ~/workspace/hsg-project
github .
cd repos
for folder in $(find * -maxdepth 0 -type d ); do github $folder; done
View production-status.xq
xquery version "3.1";
let $vols-in-production := collection('/db/apps/frus/bibliography')/volume[not(publication-status = ('published', 'being-planned-research-not-yet-begun'))]
let $admins := doc('/db/apps/frus/code-tables/administration-code-table.xml')//item
let $groupings := doc('/db/apps/frus/code-tables/grouping-code-table.xml')//item
let $statuses := doc('/db/apps/frus/code-tables/publication-status-codes.xml')//item
let $admin-years :=
map {
"truman": "1945–1953",
"eisenhower": "1953–1961",

Install saxon via homebrew to execute XQuery files on macOS

Install saxon

$ brew install saxon
$ brew info saxon
saxon: stable
XSLT and XQuery processor
View available-collation-locales.xq
xquery version "3.1";
(: List all collation locales available for use in the '?lang=' parameter of a collation URI.
: @see
: @see
element collations {
for $collation in util:collations()
order by $collation
# Add to ~/.bash_profile
# See
export JAVA_HOME=$(/usr/libexec/java_home)
View secretary-travels-just-before-inauguration.xq
xquery version "3.1";
let $all-secretary-travels := collection('/db/apps/travels/secretary-travels')//trip
let $days-before-inauguration := 8
let $duration := xs:dayTimeDuration("P" || $days-before-inauguration || "D")
let $presidents := doc("/db/apps/travels/presidents/presidents.xml")//president
for $president in $presidents
let $inauguration-date := xs:date($president/took-office-date)
let $travels-within-period-before-inauguration :=
for $trip in $all-secretary-travels
View principal-officers-and-chiefs-of-mission-from-new-jersey.xq
xquery version "3.0";
(: Displays a list of Principals & Chiefs hailing from New Jersey.
: Updated from to use new hsg3 site structure and modules
declare namespace output="";
declare option output:method "html5";
declare option output:media-type "text/html";
View test-range-index.xq
xquery version "3.1";
declare namespace tei="";
(: setup :)
let $login := xmldb:login('/db', 'admin', '')
let $test1-col := xmldb:create-collection('/db', 'test1')
let $test1-config-col := xmldb:create-collection('/db/system/config/db', 'test1')
let $test2-col := xmldb:create-collection('/db', 'test2')