Skip to content

Instantly share code, notes, and snippets.

Joe Wicentowski joewiz

Block or report user

Report or block joewiz

Hide content and notifications from this user.

Learn more about blocking users

Contact Support about this user’s behavior.

Learn more about reporting abuse

Report abuse
View GitHub Profile
@joewiz
joewiz / post-mortem.md
Last active Jul 11, 2019
Recovery from nginx "Too many open files" error on Amazon AWS Linux
View post-mortem.md

On Tue Oct 27, 2015, history.state.gov began buckling under load, intermittently issuing 500 errors. Nginx's error log was sprinkled with the following errors:

2015/10/27 21:48:36 [crit] 2475#0: accept4() failed (24: Too many open files) 2015/10/27 21:48:36 [alert] 2475#0: *7163915 socket() failed (24: Too many open files) while connecting to upstream...

An article at http://www.cyberciti.biz/faq/linux-unix-nginx-too-many-open-files/ provided directions that mostly worked. Below are the steps we followed. The steps that diverged from the article's directions are marked with an *.

    • Instead of using su to run ulimit on the nginx account, use ps aux | grep nginx to locate nginx's process IDs. Then query each process's file handle limits using cat /proc/pid/limits (where pid is the process id retrieved from ps). (Note: sudo may be necessary on your system for the cat command here, depending on your system.)
  1. Added fs.file-max = 70000 to /etc/sysctl.conf
  2. Added `nginx soft nofile 1
@joewiz
joewiz / 01-levenshtein-distance.xq
Last active Jun 4, 2019
Calculate Levenshtein Distance, using XQuery
View 01-levenshtein-distance.xq
xquery version "3.1";
(:~
Calculate Levenshtein Distance, using XQuery
@author Guillaume Mella
@see http://apps.jmmc.fr/~mellag//xquery/levenshtein/2018/01/19/xquery-levenshtein-distance/
:)
declare function local:levenshtein-distance($string1 as xs:string?, $string2 as xs:string?)
as xs:integer
@joewiz
joewiz / data-composition.xqy
Last active Jun 4, 2019 — forked from xquery/data-composition.xqy
This works in BaseX and Saxon, but eXist returns an error: `exerr:ERROR cannot convert function(*)('') to a node set [at line 59, column 15]`
View data-composition.xqy
xquery version "3.0";
(: discovered via https://twitter.com/_james_fuller/status/1087706435176288257, updated with HTTP URL for data.xq :)
(: the power of algebraic data types in xquery
This example shows how we can composite up data models
which 'carry' their own operations along with them.
:)
@joewiz
joewiz / saxon-homebrew-macos-xquery.md
Last active Jun 4, 2019
Install saxon via homebrew for command line XQuery on macOS
View saxon-homebrew-macos-xquery.md
@joewiz
joewiz / an-introduction-to-recursion-in-xquery.md
Last active Jun 1, 2019
An introduction to recursion in XQuery
View an-introduction-to-recursion-in-xquery.md

An introduction to recursion in XQuery

  • Created: Nov 28, 2017
  • Updated: Nov 29, 2017: Now covers transformation of XML documents

Recursion is a powerful programming technique, but the idea is simple: instead of performing a single operation, a function calls itself repeatedly to whittle through a larger task. In XQuery, recursion can be used to accomplish complex tasks on data that a plain FLWOR expression (which iterates through a sequence) cannot, such as transforming an entire XML document from one format into another, like TEI or DocBook into HTML, EPUB, LaTeX, or XSL-FO. Transforming a document is well-suited to recursion because each of the document's nodes may need to be examined and manipulated based on the node's type, name, and location in the document; and once a node has been processed, the transformation must continue processing the nodes' children and descendants until the deepest leaf node has been processed. But learning the technique of recursion is often hard for a beginning program

@joewiz
joewiz / json-xml.xqm
Last active May 29, 2019
An implementation of XQuery 3.1's fn:json-to-xml and fn:xml-to-json functions for eXist
View json-xml.xqm
xquery version "3.1";
(:~
: An implementation of XQuery 3.1's fn:json-to-xml and fn:xml-to-json functions for eXist, which does not support them natively as of 4.3.0.
:
: @author Joe Wicentowski
: @version 0.4
: @see http://www.w3.org/TR/xpath-functions-31/#json
:)
module namespace jx = "http://joewiz.org/ns/xquery/json-xml";
@joewiz
joewiz / yaml-to-xml.xq
Created Aug 22, 2016
Convert YAML to XML, with XQuery
View yaml-to-xml.xq
xquery version "3.0";
(: doesn't support YAML indentation yet - just a start :)
declare function local:process-yaml-value($value) {
let $single-quote := "^'(.+)'$"
let $double-quote := '^"(.+)"$'
return
if (matches($value, $single-quote) or matches($value, $double-quote)) then
let $pattern := "^['""](.+)['""]$"
@joewiz
joewiz / create-travels-tsv.xq
Last active Mar 14, 2019
Generate a TSV of Travels of President and Secretary of State
View create-travels-tsv.xq
xquery version "3.1";
(: Generate a TSV of Travels of President and Secretary of State.
: Assumes https://github.com/HistoryAtState/travels has been installed.
:)
(:~
: A function for constructing a TSV (tab-separated value) file
:
: @param A sequence of column headings
@joewiz
joewiz / create-travels-tsv.xq
Last active Mar 14, 2019
Generate a TSV of Travels of the Secretary of State
View create-travels-tsv.xq
xquery version "3.1";
(: Generate a TSV of Travels of President and Secretary of State.
: Assumes https://github.com/HistoryAtState/travels has been installed.
:)
(:~
: A function for constructing a TSV (tab-separated value) file
:
: @param A sequence of column headings
@joewiz
joewiz / xquery-3.1-boilerplate.xq
Created Mar 4, 2019
XQuery 3.1 module boilerplate
View xquery-3.1-boilerplate.xq
xquery version "3.1";
(: Standard namespaces from https://www.w3.org/TR/xpath-functions-31/#namespace-prefixes :)
declare namespace array="http://www.w3.org/2005/xpath-functions/array";
declare namespace err="http://www.w3.org/2005/xqt-errors";
declare namespace fn="http://www.w3.org/2005/xpath-functions";
declare namespace map="http://www.w3.org/2005/xpath-functions/map";
declare namespace math="http://www.w3.org/2005/xpath-functions/math";
declare namespace output="http://www.w3.org/2010/xslt-xquery-serialization";
declare namespace xs="http://www.w3.org/2001/XMLSchema";
You can’t perform that action at this time.