Skip to content

Instantly share code, notes, and snippets.

@szydell
Created February 22, 2019 07:35
Show Gist options
  • Save szydell/c2a9c01aadced506b1bfb16445a15bd1 to your computer and use it in GitHub Desktop.
Save szydell/c2a9c01aadced506b1bfb16445a15bd1 to your computer and use it in GitHub Desktop.
simdjson core
[szydell@laPtak simdjson]$ ls
amalgamation.sh AUTHORS benchmark CMakeLists.txt dependencies doc include jsonchecker jsonexamples LICENSE Makefile Notes.md README.md scripts singleheader src tape.md tests tools windows
[szydell@laPtak simdjson]$ make
g++ -std=c++17 -march=native -Wall -Wextra -Wshadow -Iinclude -Ibenchmark/linux -O3 -o parse src/jsonioutil.cpp src/jsonparser.cpp src/stage1_find_marks.cpp src/stage2_build_tape.cpp benchmark/parse.cpp
g++ -std=c++17 -march=native -Wall -Wextra -Wshadow -Iinclude -Ibenchmark/linux -O3 -o minify src/jsonminifier.cpp src/jsonioutil.cpp src/jsonparser.cpp src/stage1_find_marks.cpp src/stage2_build_tape.cpp tools/minify.cpp -I.
g++ -std=c++17 -march=native -Wall -Wextra -Wshadow -Iinclude -Ibenchmark/linux -O3 -o json2json tools/json2json.cpp src/jsonioutil.cpp src/jsonparser.cpp src/stage1_find_marks.cpp src/stage2_build_tape.cpp -I.
g++ -std=c++17 -march=native -Wall -Wextra -Wshadow -Iinclude -Ibenchmark/linux -O3 -o jsonstats tools/jsonstats.cpp src/jsonioutil.cpp src/jsonparser.cpp src/stage1_find_marks.cpp src/stage2_build_tape.cpp -I.
g++ -std=c++17 -march=native -Wall -Wextra -Wshadow -Iinclude -Ibenchmark/linux -O3 -o statisticalmodel src/jsonioutil.cpp src/jsonparser.cpp src/stage1_find_marks.cpp src/stage2_build_tape.cpp benchmark/statisticalmodel.cpp
[szydell@laPtak simdjson]$ make test
g++ -std=c++17 -march=native -Wall -Wextra -Wshadow -Iinclude -Ibenchmark/linux -O3 -o jsoncheck src/jsonioutil.cpp src/jsonparser.cpp src/stage1_find_marks.cpp src/stage2_build_tape.cpp tests/jsoncheck.cpp -I.
g++ -std=c++17 -march=native -Wall -Wextra -Wshadow -Iinclude -Ibenchmark/linux -O3 -o numberparsingcheck tests/numberparsingcheck.cpp src/jsonioutil.cpp src/jsonparser.cpp src/stage1_find_marks.cpp -I. -DJSON_TEST_NUMBERS
g++ -std=c++17 -march=native -Wall -Wextra -Wshadow -Iinclude -Ibenchmark/linux -O3 -o stringparsingcheck tests/stringparsingcheck.cpp src/jsonioutil.cpp src/jsonparser.cpp src/stage1_find_marks.cpp -I. -DJSON_TEST_STRINGS
./numberparsingcheck
Usage: ./numberparsingcheck <directorywithjsonfiles>
We are going to assume you mean to use the 'jsonchecker' and 'jsonexamples' directories.
File fail11.json is not valid --- integers: 1 floats: 0 invalid: 0 total numbers: 1
File fail13.json is not valid --- integers: 0 floats: 0 invalid: 1 total numbers: 1
File fail14.json is not valid --- integers: 0 floats: 0 invalid: 1 total numbers: 1
File fail29.json is not valid --- integers: 0 floats: 0 invalid: 1 total numbers: 1
File fail30.json is not valid --- integers: 0 floats: 0 invalid: 1 total numbers: 1
File fail31.json is not valid --- integers: 0 floats: 0 invalid: 1 total numbers: 1
File fail37.json is not valid --- integers: 1 floats: 0 invalid: 0 total numbers: 1
File fail38.json is not valid --- integers: 1 floats: 0 invalid: 0 total numbers: 1
File fail39_EXCLUDE.json is valid --- integers: 2 floats: 0 invalid: 0 total numbers: 2
Warning: foundInvalidNumber 9223372036854775808
whereas strtod parses it to 9223372036854775808.000000, while parsing jsonchecker/fail40_s64boverflow.json
File fail40_s64boverflow.json is not valid --- integers: 0 floats: 0 invalid: 1 total numbers: 1
Warning: foundInvalidNumber 18446744073709551616
whereas strtod parses it to 18446744073709551616.000000, while parsing jsonchecker/fail41_toolarge.json
File fail41_toolarge.json is not valid --- integers: 0 floats: 0 invalid: 1 total numbers: 1
File fail43.json is not valid --- integers: 0 floats: 0 invalid: 1 total numbers: 1
Warning: foundInvalidNumber -2.]
whereas strtod parses it to -2.000000, while parsing jsonchecker/fail44.json
File fail44.json is not valid --- integers: 0 floats: 0 invalid: 1 total numbers: 1
Warning: foundInvalidNumber 0.e1]
whereas strtod parses it to 0.000000, while parsing jsonchecker/fail45.json
File fail45.json is not valid --- integers: 0 floats: 0 invalid: 1 total numbers: 1
Warning: foundInvalidNumber 2.e+3]
��^�� whereas strtod parses it to 2000.000000, while parsing jsonchecker/fail46.json
File fail46.json is not valid --- integers: 0 floats: 0 invalid: 1 total numbers: 1
Warning: foundInvalidNumber 2.e-3]
whereas strtod parses it to 0.002000, while parsing jsonchecker/fail47.json
File fail47.json is not valid --- integers: 0 floats: 0 invalid: 1 total numbers: 1
Warning: foundInvalidNumber 2.e3]
whereas strtod parses it to 2000.000000, while parsing jsonchecker/fail48.json
File fail48.json is not valid --- integers: 0 floats: 0 invalid: 1 total numbers: 1
Warning: foundInvalidNumber -.123]
whereas strtod parses it to -0.123000, while parsing jsonchecker/fail49.json
File fail49.json is not valid --- integers: 0 floats: 0 invalid: 1 total numbers: 1
Warning: foundInvalidNumber 1.]
whereas strtod parses it to 1.000000, while parsing jsonchecker/fail50.json
File fail50.json is not valid --- integers: 0 floats: 0 invalid: 1 total numbers: 1
File fail55.json is not valid --- integers: 1 floats: 0 invalid: 0 total numbers: 1
File pass01.json is valid --- integers: 22 floats: 10 invalid: 0 total numbers: 32
File pass04.json is valid --- integers: 0 floats: 2 invalid: 0 total numbers: 2
File pass05.json is valid --- integers: 0 floats: 1 invalid: 0 total numbers: 1
File pass08.json is valid --- integers: 1 floats: 0 invalid: 0 total numbers: 1
File pass11.json is valid --- integers: 1 floats: 0 invalid: 0 total numbers: 1
File pass12.json is valid --- integers: 1 floats: 0 invalid: 0 total numbers: 1
File pass13.json is valid --- integers: 1 floats: 0 invalid: 0 total numbers: 1
File pass14.json is valid --- integers: 2 floats: 0 invalid: 0 total numbers: 2
All ok.
File apache_builds.json is valid --- integers: 2 floats: 0 invalid: 0 total numbers: 2
File canada.json is valid --- integers: 46 floats: 111080 invalid: 0 total numbers: 111126
File citm_catalog.json is valid --- integers: 14392 floats: 0 invalid: 0 total numbers: 14392
File github_events.json is valid --- integers: 149 floats: 0 invalid: 0 total numbers: 149
File instruments.json is valid --- integers: 4935 floats: 0 invalid: 0 total numbers: 4935
File marine_ik.json is valid --- integers: 130225 floats: 114950 invalid: 0 total numbers: 245175
File mesh.json is valid --- integers: 40613 floats: 32400 invalid: 0 total numbers: 73013
File mesh.pretty.json is valid --- integers: 40613 floats: 32400 invalid: 0 total numbers: 73013
File numbers.json is valid --- integers: 0 floats: 10001 invalid: 0 total numbers: 10001
File random.json is valid --- integers: 5002 floats: 0 invalid: 0 total numbers: 5002
File twitter.json is valid --- integers: 2108 floats: 1 invalid: 0 total numbers: 2109
File twitterescaped.json is valid --- integers: 2108 floats: 1 invalid: 0 total numbers: 2109
All ok.
free(): invalid pointer
make: *** [Makefile:60: test] Aborted (core dumped)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment