Skip to content

Instantly share code, notes, and snippets.

@sadikovi

sadikovi/Project.txt

Last active May 28, 2017 07:03
Show Gist options
  • Save sadikovi/41c07e9f76177820b7f9894c79a2efa1 to your computer and use it in GitHub Desktop.
Save sadikovi/41c07e9f76177820b7f9894c79a2efa1 to your computer and use it in GitHub Desktop.
Download ZIP
Benchmark results for Parquet, ORC and Riff (local[1], Updated as of a5e6edf4751a2f4263fcf7a52ce1809cbd321e23)
Raw
Project.txt
Java HotSpot(TM) 64-Bit Server VM 1.7.0_80-b15 on Mac OS X 10.12.4
Intel(R) Core(TM) i5-4258U CPU @ 2.40GHz
SQL project: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
Riff (all fields) 1426 / 1459 0.7 1425.7 1.0X
Riff (1 field) 876 / 983 1.1 876.3 1.6X
Riff (3 fields) 974 / 1021 1.0 974.1 1.5X
Riff (6 fields) 1354 / 1477 0.7 1353.6 1.1X
Raw
Query.txt
Java HotSpot(TM) 64-Bit Server VM 1.7.0_80-b15 on Mac OS X 10.12.4
Intel(R) Core(TM) i5-4258U CPU @ 2.40GHz
SQL Query (int filter, one record): Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
Parquet 289 / 310 3.5 289.2 1.0X
ORC 1460 / 1467 0.7 1459.7 0.2X
Riff 79 / 86 12.7 78.8 3.7X
Java HotSpot(TM) 64-Bit Server VM 1.7.0_80-b15 on Mac OS X 10.12.4
Intel(R) Core(TM) i5-4258U CPU @ 2.40GHz
SQL Query (string filter, one record): Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
Parquet 569 / 574 1.8 569.1 1.0X
ORC 1410 / 1419 0.7 1410.2 0.4X
Riff 75 / 81 13.3 75.2 7.6X
Raw
Scan.txt
Java HotSpot(TM) 64-Bit Server VM 1.7.0_80-b15 on Mac OS X 10.12.4
Intel(R) Core(TM) i5-4258U CPU @ 2.40GHz
SQL scan (no filter, all fields): Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
Parquet 1209 / 1264 0.8 1208.6 1.0X
ORC 1904 / 1958 0.5 1903.6 0.6X
Riff 1410 / 1447 0.7 1410.2 0.9X
Java HotSpot(TM) 64-Bit Server VM 1.7.0_80-b15 on Mac OS X 10.12.4
Intel(R) Core(TM) i5-4258U CPU @ 2.40GHz
SQL scan (string filter, all fields): Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
Parquet 601 / 639 1.7 601.4 1.0X
ORC 1429 / 1443 0.7 1429.0 0.4X
Riff 81 / 89 12.4 80.5 7.5X
Java HotSpot(TM) 64-Bit Server VM 1.7.0_80-b15 on Mac OS X 10.12.4
Intel(R) Core(TM) i5-4258U CPU @ 2.40GHz
SQL scan (string filter, projection): Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
Parquet 317 / 322 3.2 317.0 1.0X
ORC 601 / 612 1.7 600.6 0.5X
Riff 76 / 81 13.1 76.0 4.2X
Java HotSpot(TM) 64-Bit Server VM 1.7.0_80-b15 on Mac OS X 10.12.4
Intel(R) Core(TM) i5-4258U CPU @ 2.40GHz
SQL scan (int filter, projection): Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
Parquet 211 / 224 4.7 211.1 1.0X
ORC 407 / 414 2.5 407.4 0.5X
Riff 143 / 152 7.0 142.9 1.5X
Raw
Write.txt
Java HotSpot(TM) 64-Bit Server VM 1.7.0_80-b15 on Mac OS X 10.12.4
Intel(R) Core(TM) i5-4258U CPU @ 2.40GHz
SQL Write: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
Parquet write, gzip 16422 / 16660 0.1 16422.4 1.0X
Parquet write, snappy 8474 / 8785 0.1 8473.5 1.9X
ORC write, zlib/deflate 9791 / 10168 0.1 9791.0 1.7X
Riff write (+column filters), gzip 11984 / 12024 0.1 11984.3 1.4X
Riff write (+column filters), deflate 11891 / 11989 0.1 11890.9 1.4X
Riff write (+column filters), snappy 5601 / 5709 0.2 5601.0 2.9X
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment