Last active
December 26, 2015 00:09
-
-
Save shikhar/7062026 to your computer and use it in GitHub Desktop.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| All benchmarking was done on a Macbook Pro with a 2.4GHz Intel Core i7, SSD, and 16GB RAM using luceneutil against WIKI_MEDIUM_10M. | |
| baseline = trunk | |
| patched = https://github.com/shikhar/lucene-solr | |
| ser = as is | |
| par = parallelism hacked in to IndexSearcher constructor with a ForkJoinPool <https://gist.github.com/anonymous/7048089> | |
| # ser vs ser | |
| Report after iter 19: | |
| TaskQPS baseline StdDev QPS patched StdDev Pct diff | |
| AndHighLow 614.67 (2.0%) 603.32 (3.0%) -1.8% ( -6% - 3%) | |
| HighSloppyPhrase 10.97 (4.0%) 10.83 (3.9%) -1.3% ( -8% - 6%) | |
| OrNotHighHigh 28.78 (9.0%) 28.42 (9.4%) -1.3% ( -18% - 18%) | |
| OrNotHighLow 95.25 (8.3%) 94.12 (8.9%) -1.2% ( -17% - 17%) | |
| OrHighNotLow 75.67 (10.2%) 74.77 (10.7%) -1.2% ( -20% - 21%) | |
| OrHighNotMed 67.62 (9.6%) 66.86 (10.1%) -1.1% ( -19% - 20%) | |
| MedSloppyPhrase 21.86 (2.6%) 21.62 (2.8%) -1.1% ( -6% - 4%) | |
| OrHighHigh 43.22 (10.1%) 42.76 (10.7%) -1.1% ( -19% - 21%) | |
| OrHighLow 105.11 (9.8%) 104.03 (10.2%) -1.0% ( -19% - 21%) | |
| OrHighMed 90.10 (9.4%) 89.27 (10.0%) -0.9% ( -18% - 20%) | |
| OrHighNotHigh 33.24 (9.3%) 32.94 (9.9%) -0.9% ( -18% - 20%) | |
| PKLookup 258.80 (3.3%) 256.53 (3.2%) -0.9% ( -7% - 5%) | |
| OrNotHighMed 72.73 (8.2%) 72.09 (8.9%) -0.9% ( -16% - 17%) | |
| IntNRQ 9.83 (7.2%) 9.75 (8.5%) -0.9% ( -15% - 16%) | |
| LowSloppyPhrase 31.85 (2.6%) 31.57 (3.0%) -0.9% ( -6% - 4%) | |
| Respell 48.48 (2.8%) 48.13 (2.8%) -0.7% ( -6% - 5%) | |
| Prefix3 17.51 (4.1%) 17.40 (4.8%) -0.6% ( -9% - 8%) | |
| HighSpanNear 21.05 (4.3%) 20.94 (4.3%) -0.5% ( -8% - 8%) | |
| MedSpanNear 12.89 (4.9%) 12.83 (4.9%) -0.5% ( -9% - 9%) | |
| HighPhrase 3.31 (9.0%) 3.30 (8.3%) -0.5% ( -16% - 18%) | |
| AndHighHigh 41.38 (1.5%) 41.20 (1.7%) -0.4% ( -3% - 2%) | |
| LowPhrase 17.64 (4.9%) 17.58 (4.3%) -0.4% ( -9% - 9%) | |
| AndHighMed 229.07 (1.8%) 228.24 (1.6%) -0.4% ( -3% - 3%) | |
| LowSpanNear 32.33 (5.0%) 32.27 (4.8%) -0.2% ( -9% - 10%) | |
| Wildcard 25.98 (3.6%) 25.97 (4.0%) -0.1% ( -7% - 7%) | |
| Fuzzy1 55.58 (3.1%) 55.63 (3.6%) 0.1% ( -6% - 6%) | |
| MedPhrase 31.90 (5.7%) 31.99 (5.5%) 0.3% ( -10% - 12%) | |
| Fuzzy2 84.83 (2.7%) 85.11 (3.5%) 0.3% ( -5% - 6%) | |
| LowTerm 617.01 (3.6%) 620.03 (4.9%) 0.5% ( -7% - 9%) | |
| MedTerm 253.32 (5.5%) 257.80 (9.0%) 1.8% ( -12% - 17%) | |
| HighTerm 140.88 (6.4%) 143.94 (11.0%) 2.2% ( -14% - 20%) | |
| # ser vs par | |
| Report after iter 19: | |
| TaskQPS baseline StdDev QPS patched StdDev Pct diff | |
| Respell 47.23 (3.4%) 36.69 (4.5%) -22.3% ( -29% - -15%) | |
| MedSpanNear 201.62 (5.4%) 160.46 (9.3%) -20.4% ( -33% - -5%) | |
| PKLookup 258.74 (2.7%) 207.22 (7.2%) -19.9% ( -29% - -10%) | |
| Fuzzy2 57.72 (2.4%) 49.44 (4.6%) -14.3% ( -20% - -7%) | |
| Fuzzy1 70.74 (2.3%) 61.74 (5.4%) -12.7% ( -19% - -5%) | |
| LowSpanNear 48.66 (4.7%) 52.26 (5.7%) 7.4% ( -2% - 18%) | |
| AndHighLow 813.26 (3.1%) 1131.83 (11.9%) 39.2% ( 23% - 55%) | |
| LowTerm 661.11 (4.8%) 1069.85 (18.0%) 61.8% ( 37% - 88%) | |
| MedPhrase 108.90 (5.2%) 180.88 (12.5%) 66.1% ( 46% - 88%) | |
| MedSloppyPhrase 76.49 (3.2%) 144.18 (19.2%) 88.5% ( 64% - 114%) | |
| AndHighMed 211.07 (0.9%) 404.29 (25.0%) 91.5% ( 65% - 118%) | |
| OrNotHighLow 105.29 (6.5%) 212.92 (30.7%) 102.2% ( 61% - 148%) | |
| MedTerm 255.31 (7.5%) 522.83 (25.3%) 104.8% ( 66% - 148%) | |
| OrHighNotLow 80.56 (6.2%) 166.64 (31.0%) 106.9% ( 65% - 153%) | |
| OrHighMed 88.13 (5.7%) 185.06 (18.2%) 110.0% ( 81% - 141%) | |
| LowPhrase 65.13 (4.9%) 140.35 (20.5%) 115.5% ( 85% - 148%) | |
| OrHighLow 92.78 (5.8%) 202.49 (28.2%) 118.2% ( 79% - 161%) | |
| HighSpanNear 28.89 (5.9%) 63.13 (20.8%) 118.5% ( 86% - 154%) | |
| OrNotHighMed 106.01 (6.1%) 233.84 (29.4%) 120.6% ( 80% - 166%) | |
| OrHighNotHigh 44.38 (5.9%) 104.50 (22.1%) 135.5% ( 101% - 173%) | |
| OrHighNotMed 50.94 (5.9%) 120.33 (23.5%) 136.2% ( 100% - 176%) | |
| HighSloppyPhrase 25.72 (3.7%) 63.78 (14.2%) 148.0% ( 125% - 172%) | |
| OrNotHighHigh 42.05 (6.0%) 105.41 (23.6%) 150.7% ( 114% - 191%) | |
| OrHighHigh 26.85 (5.5%) 67.68 (23.7%) 152.1% ( 116% - 192%) | |
| Prefix3 49.32 (1.7%) 128.27 (26.7%) 160.1% ( 129% - 191%) | |
| HighTerm 38.59 (9.3%) 100.51 (11.6%) 160.5% ( 127% - 199%) | |
| Wildcard 40.01 (3.0%) 105.50 (19.5%) 163.7% ( 137% - 191%) | |
| IntNRQ 8.51 (1.6%) 22.85 (19.2%) 168.5% ( 145% - 192%) | |
| AndHighHigh 47.28 (1.1%) 127.44 (28.0%) 169.5% ( 138% - 200%) | |
| LowSloppyPhrase 29.50 (2.4%) 83.25 (17.1%) 182.2% ( 158% - 206%) | |
| HighPhrase 17.00 (6.5%) 48.24 (16.7%) 183.8% ( 150% - 221%) | |
| # par vs par | |
| Report after iter 19: | |
| TaskQPS baseline StdDev QPS patched StdDev Pct diff | |
| LowSpanNear 193.28 (8.5%) 185.88 (7.2%) -3.8% ( -18% - 13%) | |
| HighSloppyPhrase 63.36 (7.4%) 61.55 (9.7%) -2.8% ( -18% - 15%) | |
| AndHighHigh 120.77 (9.4%) 117.69 (13.2%) -2.5% ( -22% - 22%) | |
| AndHighMed 173.51 (16.1%) 169.26 (17.7%) -2.4% ( -31% - 37%) | |
| OrHighNotHigh 57.79 (10.5%) 56.64 (8.7%) -2.0% ( -19% - 19%) | |
| PKLookup 206.74 (8.6%) 203.27 (8.4%) -1.7% ( -17% - 16%) | |
| HighTerm 94.10 (7.1%) 92.64 (11.8%) -1.6% ( -19% - 18%) | |
| IntNRQ 38.99 (7.1%) 38.40 (5.7%) -1.5% ( -13% - 12%) | |
| Fuzzy1 72.31 (5.6%) 71.43 (7.9%) -1.2% ( -13% - 13%) | |
| MedPhrase 46.39 (6.6%) 45.88 (6.1%) -1.1% ( -12% - 12%) | |
| Fuzzy2 25.07 (6.2%) 24.81 (5.6%) -1.1% ( -12% - 11%) | |
| HighPhrase 42.17 (8.2%) 41.76 (8.0%) -1.0% ( -15% - 16%) | |
| MedSloppyPhrase 56.69 (6.6%) 56.16 (7.5%) -0.9% ( -14% - 14%) | |
| OrHighHigh 56.99 (10.0%) 56.47 (9.0%) -0.9% ( -18% - 20%) | |
| HighSpanNear 39.12 (5.1%) 38.78 (5.5%) -0.9% ( -10% - 10%) | |
| Prefix3 126.38 (10.3%) 125.77 (11.8%) -0.5% ( -20% - 24%) | |
| Respell 42.39 (6.0%) 42.34 (3.8%) -0.1% ( -9% - 10%) | |
| LowSloppyPhrase 116.46 (8.4%) 116.49 (9.2%) 0.0% ( -16% - 19%) | |
| Wildcard 48.34 (6.4%) 48.50 (8.8%) 0.3% ( -13% - 16%) | |
| MedSpanNear 110.42 (5.7%) 110.96 (5.3%) 0.5% ( -9% - 12%) | |
| AndHighLow 992.44 (11.1%) 998.65 (12.5%) 0.6% ( -20% - 27%) | |
| OrNotHighLow 133.71 (11.9%) 134.80 (12.3%) 0.8% ( -20% - 28%) | |
| OrHighNotMed 126.10 (9.7%) 127.92 (9.0%) 1.4% ( -15% - 22%) | |
| OrHighMed 125.81 (14.2%) 127.74 (12.9%) 1.5% ( -22% - 33%) | |
| OrHighLow 129.36 (10.3%) 131.59 (14.8%) 1.7% ( -21% - 29%) | |
| OrHighNotLow 168.50 (11.5%) 173.18 (15.6%) 2.8% ( -21% - 33%) | |
| OrNotHighHigh 101.45 (11.1%) 105.03 (11.1%) 3.5% ( -16% - 28%) | |
| LowPhrase 150.79 (11.3%) 156.52 (13.0%) 3.8% ( -18% - 31%) | |
| MedTerm 440.15 (18.5%) 459.56 (16.4%) 4.4% ( -25% - 48%) | |
| LowTerm 951.46 (17.1%) 999.99 (17.0%) 5.1% ( -24% - 47%) | |
| OrNotHighMed 119.14 (12.1%) 125.37 (11.7%) 5.2% ( -16% - 32%) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment