Skip to content

Instantly share code, notes, and snippets.

Avatar

Joe Wicentowski joewiz

View GitHub Profile
@joewiz
joewiz / post-mortem.md
Last active Jan 20, 2021
Recovery from nginx "Too many open files" error on Amazon AWS Linux
View post-mortem.md

On Tue Oct 27, 2015, history.state.gov began buckling under load, intermittently issuing 500 errors. Nginx's error log was sprinkled with the following errors:

2015/10/27 21:48:36 [crit] 2475#0: accept4() failed (24: Too many open files)

2015/10/27 21:48:36 [alert] 2475#0: *7163915 socket() failed (24: Too many open files) while connecting to upstream...

An article at http://www.cyberciti.biz/faq/linux-unix-nginx-too-many-open-files/ provided directions that mostly worked. Below are the steps we followed. The steps that diverged from the article's directions are marked with an *.

  1. * Instead of using su to run ulimit on the nginx account, use ps aux | grep nginx to locate nginx's process IDs. Then query each process's file handle limits using cat /proc/pid/limits (where pid is the process id retrieved from ps). (Note: sudo may be necessary on your system for the cat command here, depending on your system.)
  2. Added fs.file-max = 70000 to /etc/sysctl.conf
@joewiz
joewiz / json-xml.xqm
Last active Jan 19, 2021
An implementation of XQuery 3.1's fn:json-to-xml and fn:xml-to-json functions for eXist
View json-xml.xqm
xquery version "3.1";
(:~
: An implementation of XQuery 3.1's fn:json-to-xml and fn:xml-to-json functions for eXist, which does not support them natively as of 4.3.0.
:
: @author Joe Wicentowski
: @version 0.4
: @see http://www.w3.org/TR/xpath-functions-31/#json
:)
module namespace jx = "http://joewiz.org/ns/xquery/json-xml";
@joewiz
joewiz / environment-variables.xq
Last active Jan 15, 2021
Display all environment variables and their values
View environment-variables.xq
xquery version "3.1";
element environment-variables {
for $var in available-environment-variables()
order by $var
return
element environment-variable {
attribute name { $var },
attribute value { environment-variable($var) }
}
@joewiz
joewiz / strip-diacritics.xq
Last active Jan 15, 2021
Strip diacritics, with XQuery
View strip-diacritics.xq
xquery version "3.1";
declare function local:strip-diacritics($string as xs:string) as xs:string {
let $normalized := normalize-unicode($string, 'NFD')
let $stripped := replace($normalized, '\p{IsCombiningDiacriticalMarks}', '')
return
$stripped
};
declare function local:inspect-diacritics($string as xs:string) as element() {
@joewiz
joewiz / date-parser.xqm
Created Aug 26, 2018
Parse various formats of date strings, in XQuery
View date-parser.xqm
xquery version "3.1";
(:
Various Date String Parser
- Parses various flavors of date strings, returns as xs:dateTime or xs:date
- Key functions: dates:parseDateTime() and dates:parseDate()
- Adapted by Joe Wicentowski from
https://github.com/marklogic-community/commons/blob/master/dates/date-parser.xqy
- Adapted to standard XQuery (instead of the MarkLogic 0.9-ml flavor)
- TODO: test against https://github.com/marklogic-community/commons/blob/master/dates/date-parser-tests.xqy
@joewiz
joewiz / fib-exist.xq
Last active Oct 10, 2020 — forked from apb2006/fib.xq
XQuery tail recursive Fibonacci function, with timing, for eXist-db
View fib-exist.xq
xquery version "3.1";
(: forked from https://gist.github.com/apb2006/4eef5889017be4a50685a467b2754d27
: with tests returned in the style of https://so.nwalsh.com/2020/10/09-fib :)
declare function local:fib($n as xs:integer, $a as xs:integer, $b as xs:integer){
switch ($n)
case 0 return $a
case 1 return $b
default return local:fib($n - 1, $b, $a + $b)
@joewiz
joewiz / saxon-homebrew-macos-xquery.md
Last active Sep 25, 2020
Install saxon via homebrew for command line XQuery on macOS
View saxon-homebrew-macos-xquery.md
@joewiz
joewiz / get-latest-created-document.xq
Last active Sep 21, 2020
Get the most recently created document in an eXist collection, using XQuery
View get-latest-created-document.xq
xquery version "3.1";
(: See discussion at http://markmail.org/message/hpu7toznx3fvdiei :)
import module namespace util="http://exist-db.org/xquery/util";
import module namespace xmldb="http://exist-db.org/xquery/xmldb";
declare function local:get-latest-created-document($collection-uri as xs:string) as map(*) {
if (xmldb:collection-available($collection-uri)) then
let $documents := xmldb:xcollection($collection-uri) ! util:document-name(.)
@joewiz
joewiz / export-eXide-tabs.xq
Last active Sep 18, 2020
Save eXide editor tabs to disk
View export-eXide-tabs.xq
xquery version "3.1";
(:
# Save eXide editor tabs to disk
1. Install "LocalStorage Manager" Chrome extension
https://chrome.google.com/webstore/detail/localstorage-manager/fkhoimdhngkiicbjobkinobjkoefhkap
@joewiz
joewiz / map-find.xqm
Last active Jun 10, 2020
An implementation of XQuery 3.1's map:find function for eXist
View map-find.xqm
xquery version "3.1";
(:~
: An implementation of XQuery 3.1's map:find function for eXist, which does not support it natively as of 3.4.0.
:
: @author Joe Wicentowski
: @see https://www.w3.org/TR/xpath-functions-31/#func-map-find
:)
module namespace mf="http://joewiz.org/ns/xquery/map-find";