Skip to content

Instantly share code, notes, and snippets.

View mswilson's full-sized avatar

mswilson

36 followers · 1 following
  • Amazon Web Services
View GitHub Profile
@mswilson
mswilson / gist:5446082
Created April 23, 2013 18:25 — forked from anonymous/gist:5446069
<h1>Xen.org Security Problem Response Process</h1>
<h2>Introduction</h2>
<p>Computer systems have bugs. Currently recognised best practice for bugs
with security implications is to notify significant downstream users in
private; leave a reasonable interval for downstreams to respond and prepare
updated software packages; then make public disclosure.</p>
<p>We want to encourage people to report bugs they find to us. Therefore we
will treat with respect the requests of discoverers, or other vendors, who
report problems to us.</p>
@mswilson
mswilson / Early-AWS-Hadoop-Contributions.md
Created December 7, 2018 07:14
Early AWS Hadoop contributions
@mswilson
mswilson / gist:80a60768de0df1873d45d3e4f31f2f4b
Created March 14, 2019 17:25
msw@carbon:~/git/elasticsearch$ ./gradlew -p distribution/packages/oss-rpm assemble --parallel
Starting a Gradle Daemon, 1 busy Daemon could not be reused, use --status for details
FAILURE: Build failed with an exception.
* Where:
Build file '/home/msw/git/elasticsearch/build.gradle' line: 269
* What went wrong:
A problem occurred configuring root project 'elasticsearch'.
@mswilson
mswilson / results.txt
Created December 8, 2019 22:35
simdjson parser.sh run on m6g.16xlarge on Ubuntu Bionic ami-03f3d98d9ddb32a79 out-of-the-box
ubuntu@ip-10-0-2-180:~/simdjson/build$ ../scripts/parser.sh
make: 'parsingcompetition' is up to date.
/home/ubuntu/simdjson/scripts/../jsonexamples/apache_builds.json
getline : 0.001 (clock units) per input byte (best) 0.001 (clock units) per input byte (avg) 0.769 GB/s (error margin: 0.003 GB/s)
simdjson (dynamic mem) : 0.001 (clock units) per input byte (best) 0.001 (clock units) per input byte (avg) 1.062 GB/s (error margin: 0.011 GB/s)
simdjson : 0.001 (clock units) per input byte (best) 0.001 (clock units) per input byte (avg) 1.047 GB/s (error margin: 0.010 GB/s)
RapidJSON : 0.004 (clock units) per input byte (best) 0.004 (clock units) per input byte (avg) 0.269 GB/s (error margin: 0.014 GB/s)
RapidJSON (insitu) : 0.004 (clock units) per input byte (best) 0.004 (clock units) per input byte (avg) 0.260 GB
@mswilson
mswilson / force-no-AVX2.txt
Created December 9, 2019 00:07
simdjson parser.sh run on m5.12xlarge on Ubuntu Bionic ami-00a208c7cdba991ea
ubuntu@ip-10-0-2-210:~/simdjson/benchmark$ ../scripts/parser.sh
In case of build error due to missing files, try 'make clean'
g++ -msse4.2 -mpclmul -std=c++17 -pthread -Wall -Wextra -Wshadow -Iinclude -Isrc -Ibenchmark/linux -O3 -o parsingcompetition src/jsonioutil.cpp src/jsonparser.cpp src/jsonstream.cpp src/simdjson.cpp src/stage1_find_marks.cpp src/stage2_build_tape.cpp src/parsedjson.cpp src/parsedjsoniterator.cpp benchmark/parsingcompetition.cpp -I. -Idependencies/json/single_include -Idependencies/rapidjson/include -Idependencies/sajson/include -Idependencies/cJSON -Idependencies/jsmn
In file included from benchmark/parsingcompetition.cpp:22:0:
dependencies/sajson/include/sajson.h: In member function ‘sajson::dynamic_allocation::allocator sajson::dynamic_allocation::make_allocator(size_t, bool*) const’:
dependencies/sajson/include/sajson.h:1313:41: warning: unused parameter ‘input_document_size_in_bytes’ [-Wunused-parameter]
allocator make_allocator(size_t input_document_size_in_bytes,
@mswilson
mswilson / results.txt
Last active December 9, 2019 07:03
simdjson parser.sh run on m6g.16xlarge on Ubuntu Bionic ami-03f3d98d9ddb32a79, slightly tuned
# compiled with Cxx=g++-9 ARCHFLAGS="-march=armv8.5-a+crc -mtune=neoverse-n1 -ffast-math"
# switched from utf8_fastvalidate_algorithm.h to utf8_lookup_algorithm.h
jsonexamples/apache_builds.json
getline : 0.001 (clock units) per input byte (best) 0.001 (clock units) per input byte (avg) 0.816 GB/s (error margin: 0.004 GB/s)
simdjson (dynamic mem) : 0.001 (clock units) per input byte (best) 0.001 (clock units) per input byte (avg) 1.102 GB/s (error margin: 0.007 GB/s)
simdjson : 0.001 (clock units) per input byte (best) 0.001 (clock units) per input byte (avg) 1.100 GB/s (error margin: 0.011 GB/s)
RapidJSON : 0.003 (clock units) per input byte (best) 0.004 (clock units) per input byte (avg) 0.304 GB/s (error margin: 0.021 GB/s)
RapidJSON (insitu) : 0.003 (clock units) per input byte (best) 0.003 (clock units) per
@mswilson
mswilson / dmesg.txt
Created December 28, 2019 02:39
NetBSD 9.0_RC1 dmesg on m6g.16xlarge
[ 1.000000] Copyright (c) 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005,
[ 1.000000] 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016, 2017,
[ 1.000000] 2018, 2019 The NetBSD Foundation, Inc. All rights reserved.
[ 1.000000] Copyright (c) 1982, 1986, 1989, 1991, 1993
[ 1.000000] The Regents of the University of California. All rights reserved.
[ 1.000000] NetBSD 9.0_RC1 (GENERIC64) #0: Wed Nov 27 16:14:52 UTC 2019
[ 1.000000] mkrepro@mkrepro.NetBSD.org:/usr/src/sys/arch/evbarm/compile/GENERIC64
[ 1.000000] total memory = 250 GB
[ 1.000000] avail memory = 241 GB