jakelandis/ingest_grok_vs_dissect

## ingest_grok_vs_dissect
Test setup: https://gist.github.com/jakelandis/7b0f5511c714e7f0e0f4f9a7b9b1fac8


Run on this machine:
```
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              8
On-line CPU(s) list: 0-7
Thread(s) per core:  2
Core(s) per socket:  4
Socket(s):           1
NUMA node(s):        1
Vendor ID:           GenuineIntel
CPU family:          6
Model:               158
Model name:          Intel(R) Core(TM) i7-8705G CPU @ 3.10GHz
Stepping:            9
CPU MHz:             800.059
CPU max MHz:         4100.0000
CPU min MHz:         800.0000
BogoMIPS:            6192.00
Virtualization:      VT-x
L1d cache:           32K
L1i cache:           32K
L2 cache:            256K
L3 cache:            8192K
NUMA node0 CPU(s):   0-7
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf tsc_known_freq pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb invpcid_single pti ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid mpx rdseed adx smap clflushopt intel_pt xsaveopt xsavec xgetbv1 xsaves dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp
```
./gradlew :benchmarks:jmh
To honour the JVM settings for this build a new JVM will be forked. Please consider using the daemon: https://docs.gradle.org/4.9/userguide/gradle_daemon.html.
Daemon will be stopped at the end of the build stopping after processing

> Configure project :benchmarks
=======================================
Elasticsearch Build Hamster says Hello!
=======================================
  Gradle Version        : 4.9
  OS Info               : Linux 4.17.4-200.fc28.x86_64 (amd64)
  JDK Version (gradle)  : Oracle Corporation 10.0.1 [Java HotSpot(TM) 64-Bit Server VM 10.0.1+10]
  JAVA_HOME (gradle)    : /home/jake/java/jdk-10.0.1
  JDK Version (compile) : Oracle Corporation 10.0.1 [Java HotSpot(TM) 64-Bit Server VM 10.0.1+10]
  JAVA_HOME (compile)   : /home/jake/java/jdk-10.0.1
  JDK Version (runtime) : Oracle Corporation 1.8.0_171 [Java HotSpot(TM) 64-Bit Server VM 25.171-b11]
  JAVA_HOME (runtime)   : /home/jake/java/jdk1.8.0_171
  Random Testing Seed   : 5C2DC2630111BAF7

> Task :benchmarks:runShadow
# JMH version: 1.19
# VM version: JDK 1.8.0_171, VM 25.171-b11
# VM invoker: /home/jake/java/jdk1.8.0_171/jre/bin/java
# VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
# Warmup: 3 iterations, 5 s each
# Measurement: 5 iterations, 30 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Throughput, ops/time
# Benchmark: org.elasticsearch.benchmark.ingest.dissect.DissectBenchmark.dissectApacheLog

# Run progress: 0.00% complete, ETA 00:22:00
# Fork: 1 of 2
# Warmup Iteration   1: 382131.484 ops/s
# Warmup Iteration   2: 422942.757 ops/s
# Warmup Iteration   3: 420766.789 ops/s
Iteration   1: 420238.856 ops/s35s]
Iteration   2: 422288.780 ops/s1m 5s]
Iteration   3: 422859.314 ops/s1m 35s]
Iteration   4: 422537.192 ops/s2m 5s]
Iteration   5: 422536.911 ops/s2m 35s]

# Run progress: 12.50% complete, ETA 00:19:17
# Fork: 2 of 2
# Warmup Iteration   1: 393997.462 ops/s
# Warmup Iteration   2: 426545.317 ops/s
# Warmup Iteration   3: 425716.391 ops/s
Iteration   1: 425035.114 ops/s3m 20s]
Iteration   2: 425499.997 ops/s3m 50s]
Iteration   3: 425379.055 ops/s4m 20s]
Iteration   4: 425164.412 ops/s4m 50s]
Iteration   5: 425090.468 ops/s5m 20s]


Result "org.elasticsearch.benchmark.ingest.dissect.DissectBenchmark.dissectApacheLog":
  423663.010 ±(99.9%) 2727.341 ops/s [Average]
  (min, avg, max) = (420238.856, 423663.010, 425499.997), stdev = 1803.967
  CI (99.9%): [420935.669, 426390.351] (assumes normal distribution)


# JMH version: 1.19
# VM version: JDK 1.8.0_171, VM 25.171-b11
# VM invoker: /home/jake/java/jdk1.8.0_171/jre/bin/java
# VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
# Warmup: 3 iterations, 5 s each
# Measurement: 5 iterations, 30 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Throughput, ops/time
# Benchmark: org.elasticsearch.benchmark.ingest.dissect.DissectBenchmark.dissectSysLog

# Run progress: 25.00% complete, ETA 00:16:31
# Fork: 1 of 2
# Warmup Iteration   1: 791042.486 ops/s
# Warmup Iteration   2: 856010.226 ops/s
# Warmup Iteration   3: 855722.273 ops/s
Iteration   1: 856413.037 ops/s6m 6s]
Iteration   2: 856822.773 ops/s6m 36s]
Iteration   3: 857616.654 ops/s7m 6s]
Iteration   4: 856052.129 ops/s7m 36s]
Iteration   5: 856967.286 ops/s8m 6s]

# Run progress: 37.50% complete, ETA 00:13:46
# Fork: 2 of 2
# Warmup Iteration   1: 761555.890 ops/s
# Warmup Iteration   2: 867385.934 ops/s
# Warmup Iteration   3: 866532.013 ops/s
Iteration   1: 867193.000 ops/s8m 51s]
Iteration   2: 867374.136 ops/s9m 21s]
Iteration   3: 867746.237 ops/s9m 51s]
Iteration   4: 867281.219 ops/s10m 21s]
Iteration   5: 868271.135 ops/s10m 51s]


Result "org.elasticsearch.benchmark.ingest.dissect.DissectBenchmark.dissectSysLog":
  862173.761 ±(99.9%) 8636.887 ops/s [Average]
  (min, avg, max) = (856052.129, 862173.761, 868271.135), stdev = 5712.766
  CI (99.9%): [853536.874, 870810.647] (assumes normal distribution)


# JMH version: 1.19
# VM version: JDK 1.8.0_171, VM 25.171-b11
# VM invoker: /home/jake/java/jdk1.8.0_171/jre/bin/java
# VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
# Warmup: 3 iterations, 5 s each
# Measurement: 5 iterations, 30 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Throughput, ops/time
# Benchmark: org.elasticsearch.benchmark.ingest.dissect.DissectBenchmark.grokApacheLog

# Run progress: 50.00% complete, ETA 00:11:01
# Fork: 1 of 2
# Warmup Iteration   1: 87336.279 ops/s
# Warmup Iteration   2: 92581.534 ops/s
# Warmup Iteration   3: 92885.364 ops/s
Iteration   1: 92499.273 ops/s[11m 36s]
Iteration   2: 92584.485 ops/s[12m 6s]
Iteration   3: 92546.039 ops/s[12m 36s]
Iteration   4: 92528.705 ops/s[13m 6s]
Iteration   5: 92546.043 ops/s[13m 36s]

# Run progress: 62.50% complete, ETA 00:08:15
# Fork: 2 of 2
# Warmup Iteration   1: 86209.457 ops/s
# Warmup Iteration   2: 89736.468 ops/s
# Warmup Iteration   3: 89811.713 ops/s
Iteration   1: 89885.211 ops/s[14m 22s]
Iteration   2: 89777.384 ops/s[14m 52s]
Iteration   3: 89845.731 ops/s[15m 22s]
Iteration   4: 89808.322 ops/s[15m 52s]
Iteration   5: 89763.499 ops/s[16m 22s]


Result "org.elasticsearch.benchmark.ingest.dissect.DissectBenchmark.grokApacheLog":
  91178.469 ±(99.9%) 2172.043 ops/s [Average]
  (min, avg, max) = (89763.499, 91178.469, 92584.485), stdev = 1436.672
  CI (99.9%): [89006.426, 93350.512] (assumes normal distribution)


# JMH version: 1.19
# VM version: JDK 1.8.0_171, VM 25.171-b11
# VM invoker: /home/jake/java/jdk1.8.0_171/jre/bin/java
# VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
# Warmup: 3 iterations, 5 s each
# Measurement: 5 iterations, 30 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Throughput, ops/time
# Benchmark: org.elasticsearch.benchmark.ingest.dissect.DissectBenchmark.grokSysLog

# Run progress: 75.00% complete, ETA 00:05:30
# Fork: 1 of 2
# Warmup Iteration   1: 283206.954 ops/s
# Warmup Iteration   2: 298554.229 ops/s
# Warmup Iteration   3: 298094.503 ops/s
Iteration   1: 298280.199 ops/s17m 7s]
Iteration   2: 298203.815 ops/s17m 37s]
Iteration   3: 298186.791 ops/s18m 7s]
Iteration   4: 298115.669 ops/s18m 37s]
Iteration   5: 298012.004 ops/s19m 7s]

# Run progress: 87.50% complete, ETA 00:02:45
# Fork: 2 of 2
# Warmup Iteration   1: 287000.123 ops/s
# Warmup Iteration   2: 303352.340 ops/s
# Warmup Iteration   3: 302562.261 ops/s

Iteration   1: 302568.397 ops/s19m 52s]
Iteration   2: 302596.566 ops/s20m 22s]
Iteration   3: 302716.606 ops/s20m 52s]
Iteration   4: 302710.091 ops/s21m 22s]
Iteration   5: 302697.268 ops/s21m 52s]


Result "org.elasticsearch.benchmark.ingest.dissect.DissectBenchmark.grokSysLog":
  300408.741 ±(99.9%) 3586.303 ops/s [Average]
  (min, avg, max) = (298012.004, 300408.741, 302716.606), stdev = 2372.118
  CI (99.9%): [296822.437, 303995.044] (assumes normal distribution)


# Run complete. Total time: 00:22:02

Benchmark                           Mode  Cnt       Score      Error  Units
DissectBenchmark.dissectApacheLog  thrpt   10  423663.010 ± 2727.341  ops/s
DissectBenchmark.dissectSysLog     thrpt   10  862173.761 ± 8636.887  ops/s
DissectBenchmark.grokApacheLog     thrpt   10   91178.469 ± 2172.043  ops/s
DissectBenchmark.grokSysLog        thrpt   10  300408.741 ± 3586.303  ops/s
	Test setup: https://gist.github.com/jakelandis/7b0f5511c714e7f0e0f4f9a7b9b1fac8


	Run on this machine:
	```
	Architecture: x86_64
	CPU op-mode(s): 32-bit, 64-bit
	Byte Order: Little Endian
	CPU(s): 8
	On-line CPU(s) list: 0-7
	Thread(s) per core: 2
	Core(s) per socket: 4
	Socket(s): 1
	NUMA node(s): 1
	Vendor ID: GenuineIntel
	CPU family: 6
	Model: 158
	Model name: Intel(R) Core(TM) i7-8705G CPU @ 3.10GHz
	Stepping: 9
	CPU MHz: 800.059
	CPU max MHz: 4100.0000
	CPU min MHz: 800.0000
	BogoMIPS: 6192.00
	Virtualization: VT-x
	L1d cache: 32K
	L1i cache: 32K
	L2 cache: 256K
	L3 cache: 8192K
	NUMA node0 CPU(s): 0-7
	Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf tsc_known_freq pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb invpcid_single pti ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid mpx rdseed adx smap clflushopt intel_pt xsaveopt xsavec xgetbv1 xsaves dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp
	```
	./gradlew :benchmarks:jmh
	To honour the JVM settings for this build a new JVM will be forked. Please consider using the daemon: https://docs.gradle.org/4.9/userguide/gradle_daemon.html.
	Daemon will be stopped at the end of the build stopping after processing

	> Configure project :benchmarks
	=======================================
	Elasticsearch Build Hamster says Hello!
	=======================================
	Gradle Version : 4.9
	OS Info : Linux 4.17.4-200.fc28.x86_64 (amd64)
	JDK Version (gradle) : Oracle Corporation 10.0.1 [Java HotSpot(TM) 64-Bit Server VM 10.0.1+10]
	JAVA_HOME (gradle) : /home/jake/java/jdk-10.0.1
	JDK Version (compile) : Oracle Corporation 10.0.1 [Java HotSpot(TM) 64-Bit Server VM 10.0.1+10]
	JAVA_HOME (compile) : /home/jake/java/jdk-10.0.1
	JDK Version (runtime) : Oracle Corporation 1.8.0_171 [Java HotSpot(TM) 64-Bit Server VM 25.171-b11]
	JAVA_HOME (runtime) : /home/jake/java/jdk1.8.0_171
	Random Testing Seed : 5C2DC2630111BAF7

	> Task :benchmarks:runShadow
	# JMH version: 1.19
	# VM version: JDK 1.8.0_171, VM 25.171-b11
	# VM invoker: /home/jake/java/jdk1.8.0_171/jre/bin/java
	# VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
	# Warmup: 3 iterations, 5 s each
	# Measurement: 5 iterations, 30 s each
	# Timeout: 10 min per iteration
	# Threads: 1 thread, will synchronize iterations
	# Benchmark mode: Throughput, ops/time
	# Benchmark: org.elasticsearch.benchmark.ingest.dissect.DissectBenchmark.dissectApacheLog

	# Run progress: 0.00% complete, ETA 00:22:00
	# Fork: 1 of 2
	# Warmup Iteration 1: 382131.484 ops/s
	# Warmup Iteration 2: 422942.757 ops/s
	# Warmup Iteration 3: 420766.789 ops/s
	Iteration 1: 420238.856 ops/s35s]
	Iteration 2: 422288.780 ops/s1m 5s]
	Iteration 3: 422859.314 ops/s1m 35s]
	Iteration 4: 422537.192 ops/s2m 5s]
	Iteration 5: 422536.911 ops/s2m 35s]

	# Run progress: 12.50% complete, ETA 00:19:17
	# Fork: 2 of 2
	# Warmup Iteration 1: 393997.462 ops/s
	# Warmup Iteration 2: 426545.317 ops/s
	# Warmup Iteration 3: 425716.391 ops/s
	Iteration 1: 425035.114 ops/s3m 20s]
	Iteration 2: 425499.997 ops/s3m 50s]
	Iteration 3: 425379.055 ops/s4m 20s]
	Iteration 4: 425164.412 ops/s4m 50s]
	Iteration 5: 425090.468 ops/s5m 20s]


	Result "org.elasticsearch.benchmark.ingest.dissect.DissectBenchmark.dissectApacheLog":
	423663.010 ±(99.9%) 2727.341 ops/s [Average]
	(min, avg, max) = (420238.856, 423663.010, 425499.997), stdev = 1803.967
	CI (99.9%): [420935.669, 426390.351] (assumes normal distribution)


	# JMH version: 1.19
	# VM version: JDK 1.8.0_171, VM 25.171-b11
	# VM invoker: /home/jake/java/jdk1.8.0_171/jre/bin/java
	# VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
	# Warmup: 3 iterations, 5 s each
	# Measurement: 5 iterations, 30 s each
	# Timeout: 10 min per iteration
	# Threads: 1 thread, will synchronize iterations
	# Benchmark mode: Throughput, ops/time
	# Benchmark: org.elasticsearch.benchmark.ingest.dissect.DissectBenchmark.dissectSysLog

	# Run progress: 25.00% complete, ETA 00:16:31
	# Fork: 1 of 2
	# Warmup Iteration 1: 791042.486 ops/s
	# Warmup Iteration 2: 856010.226 ops/s
	# Warmup Iteration 3: 855722.273 ops/s
	Iteration 1: 856413.037 ops/s6m 6s]
	Iteration 2: 856822.773 ops/s6m 36s]
	Iteration 3: 857616.654 ops/s7m 6s]
	Iteration 4: 856052.129 ops/s7m 36s]
	Iteration 5: 856967.286 ops/s8m 6s]

	# Run progress: 37.50% complete, ETA 00:13:46
	# Fork: 2 of 2
	# Warmup Iteration 1: 761555.890 ops/s
	# Warmup Iteration 2: 867385.934 ops/s
	# Warmup Iteration 3: 866532.013 ops/s
	Iteration 1: 867193.000 ops/s8m 51s]
	Iteration 2: 867374.136 ops/s9m 21s]
	Iteration 3: 867746.237 ops/s9m 51s]
	Iteration 4: 867281.219 ops/s10m 21s]
	Iteration 5: 868271.135 ops/s10m 51s]


	Result "org.elasticsearch.benchmark.ingest.dissect.DissectBenchmark.dissectSysLog":
	862173.761 ±(99.9%) 8636.887 ops/s [Average]
	(min, avg, max) = (856052.129, 862173.761, 868271.135), stdev = 5712.766
	CI (99.9%): [853536.874, 870810.647] (assumes normal distribution)


	# JMH version: 1.19
	# VM version: JDK 1.8.0_171, VM 25.171-b11
	# VM invoker: /home/jake/java/jdk1.8.0_171/jre/bin/java
	# VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
	# Warmup: 3 iterations, 5 s each
	# Measurement: 5 iterations, 30 s each
	# Timeout: 10 min per iteration
	# Threads: 1 thread, will synchronize iterations
	# Benchmark mode: Throughput, ops/time
	# Benchmark: org.elasticsearch.benchmark.ingest.dissect.DissectBenchmark.grokApacheLog

	# Run progress: 50.00% complete, ETA 00:11:01
	# Fork: 1 of 2
	# Warmup Iteration 1: 87336.279 ops/s
	# Warmup Iteration 2: 92581.534 ops/s
	# Warmup Iteration 3: 92885.364 ops/s
	Iteration 1: 92499.273 ops/s[11m 36s]
	Iteration 2: 92584.485 ops/s[12m 6s]
	Iteration 3: 92546.039 ops/s[12m 36s]
	Iteration 4: 92528.705 ops/s[13m 6s]
	Iteration 5: 92546.043 ops/s[13m 36s]

	# Run progress: 62.50% complete, ETA 00:08:15
	# Fork: 2 of 2
	# Warmup Iteration 1: 86209.457 ops/s
	# Warmup Iteration 2: 89736.468 ops/s
	# Warmup Iteration 3: 89811.713 ops/s
	Iteration 1: 89885.211 ops/s[14m 22s]
	Iteration 2: 89777.384 ops/s[14m 52s]
	Iteration 3: 89845.731 ops/s[15m 22s]
	Iteration 4: 89808.322 ops/s[15m 52s]
	Iteration 5: 89763.499 ops/s[16m 22s]


	Result "org.elasticsearch.benchmark.ingest.dissect.DissectBenchmark.grokApacheLog":
	91178.469 ±(99.9%) 2172.043 ops/s [Average]
	(min, avg, max) = (89763.499, 91178.469, 92584.485), stdev = 1436.672
	CI (99.9%): [89006.426, 93350.512] (assumes normal distribution)


	# JMH version: 1.19
	# VM version: JDK 1.8.0_171, VM 25.171-b11
	# VM invoker: /home/jake/java/jdk1.8.0_171/jre/bin/java
	# VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
	# Warmup: 3 iterations, 5 s each
	# Measurement: 5 iterations, 30 s each
	# Timeout: 10 min per iteration
	# Threads: 1 thread, will synchronize iterations
	# Benchmark mode: Throughput, ops/time
	# Benchmark: org.elasticsearch.benchmark.ingest.dissect.DissectBenchmark.grokSysLog

	# Run progress: 75.00% complete, ETA 00:05:30
	# Fork: 1 of 2
	# Warmup Iteration 1: 283206.954 ops/s
	# Warmup Iteration 2: 298554.229 ops/s
	# Warmup Iteration 3: 298094.503 ops/s
	Iteration 1: 298280.199 ops/s17m 7s]
	Iteration 2: 298203.815 ops/s17m 37s]
	Iteration 3: 298186.791 ops/s18m 7s]
	Iteration 4: 298115.669 ops/s18m 37s]
	Iteration 5: 298012.004 ops/s19m 7s]

	# Run progress: 87.50% complete, ETA 00:02:45
	# Fork: 2 of 2
	# Warmup Iteration 1: 287000.123 ops/s
	# Warmup Iteration 2: 303352.340 ops/s
	# Warmup Iteration 3: 302562.261 ops/s

	Iteration 1: 302568.397 ops/s19m 52s]
	Iteration 2: 302596.566 ops/s20m 22s]
	Iteration 3: 302716.606 ops/s20m 52s]
	Iteration 4: 302710.091 ops/s21m 22s]
	Iteration 5: 302697.268 ops/s21m 52s]


	Result "org.elasticsearch.benchmark.ingest.dissect.DissectBenchmark.grokSysLog":
	300408.741 ±(99.9%) 3586.303 ops/s [Average]
	(min, avg, max) = (298012.004, 300408.741, 302716.606), stdev = 2372.118
	CI (99.9%): [296822.437, 303995.044] (assumes normal distribution)


	# Run complete. Total time: 00:22:02

	Benchmark Mode Cnt Score Error Units
	DissectBenchmark.dissectApacheLog thrpt 10 423663.010 ± 2727.341 ops/s
	DissectBenchmark.dissectSysLog thrpt 10 862173.761 ± 8636.887 ops/s
	DissectBenchmark.grokApacheLog thrpt 10 91178.469 ± 2172.043 ops/s
	DissectBenchmark.grokSysLog thrpt 10 300408.741 ± 3586.303 ops/s