Skip to content

Instantly share code, notes, and snippets.

View jakelandis's full-sized avatar

Jake Landis jakelandis

31 followers · 9 following
View GitHub Profile
@jakelandis
jakelandis / logstash_jruby_memory_OQL
Created February 5, 2018 16:19
var countByValue = {};
var sizeByValue = {};
// Scroll the strings
heap.forEachObject(
function(strObject) {
var key = strObject.metaClass.realClass.cachedName;
if (key) {
key = key.toString();
} else {
@jakelandis
jakelandis / gist:7b0f5511c714e7f0e0f4f9a7b9b1fac8
Created July 26, 2018 17:08
//example log lines pulled from ingest grok tests: https://github.com/elastic/elasticsearch/blob/master/libs/grok/src/test/java/org/elasticsearch/grok/GrokTests.java
//validated that dissect matches via it's unit tests
@BenchmarkMode(Mode.Throughput)
@OutputTimeUnit(TimeUnit.SECONDS)
@Warmup(iterations = 3, time = 5, timeUnit = TimeUnit.SECONDS)
@Measurement(iterations = 5, time = 30, timeUnit = TimeUnit.SECONDS)
@Threads(1)
@Fork(2)
public class DissectBenchmark {
@jakelandis
jakelandis / ingest_grok_vs_dissect
Last active July 26, 2018 17:22
Test setup: https://gist.github.com/jakelandis/7b0f5511c714e7f0e0f4f9a7b9b1fac8
Run on this machine:
```
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 8
On-line CPU(s) list: 0-7
@jakelandis
jakelandis / gist:de554f394e5455ee66797c2b956d63bf
Last active August 15, 2018 18:54
rally dissect benchmarks
esrally --track=http_logs_for_ingest --track-repository=ingest --challenge=baseline --laps=1 --report-file=~/baseline1.out;\
> esrally --track=http_logs_for_ingest --track-repository=ingest --challenge=grok --laps=1; --report-file=~/grok1.out;\
> esrally --track=http_logs_for_ingest --track-repository=ingest --challenge=dissect --laps=1; --report-file=~/dissect1.out;\
> esrally --track=http_logs_for_ingest --track-repository=ingest --challenge=baseline --laps=2; --report-file=~/baseline2.out;\
> esrally --track=http_logs_for_ingest --track-repository=ingest --challenge=grok --laps=2; --report-file=~/grok2.out;\
> esrally --track=http_logs_for_ingest --track-repository=ingest --challenge=dissect --laps=2; --report-file=~/dissect2.out;\
> esrally --track=http_logs_for_ingest --track-repository=ingest --challenge=baseline --laps=3; --report-file=~/baseline3.out;\
> esrally --track=http_logs_for_ingest --track-repository=ingest --challenge=grok --laps=3; --report-file=~/grok3.out;\
> esrally --track=http_logs_for
@jakelandis
jakelandis / gist:dd0b627029f26fd52957a00b0c7e118f
Created August 15, 2018 19:56
test
@jakelandis
jakelandis / error1.md
Last active September 5, 2018 13:45 
./gradlew :x-pack:qa:third-party:active-directory:test  -Dtests.class=org.elasticsearch.xpack.security.authc.ldap.MultipleAdRealmIT -Dtests.method="testCanAuthenticateAgainstBothRealms" -Dtests.security.manager=true -Dtests.locale=en-IE -Dtests.timezone=Asia/Amman -Dcompiler.java=10 -Druntime.java=8

> Task :x-pack:qa:third-party:active-directory:compileTestJava
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: /home/jake/workspace/elasticsearch/x-pack/qa/third-party/active-directory/src/test/java/org/elasticsearch/xpack/security/authc/ldap/ADLdapUserSearchSessionFactoryTests.java uses unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
@jakelandis
jakelandis / vis_issue
Created November 7, 2018 16:54
[
{
"_id": "750a9ed0-e2ad-11e8-a029-39f86895eb73",
"_type": "visualization",
"_source": {
"title": "painless_issue",
"visState": "{\"title\":\"painless_issue\",\"type\":\"metrics\",\"params\":{\"id\":\"61ca57f0-469d-11e7-af02-69e470af7417\",\"type\":\"timeseries\",\"series\":[{\"id\":\"61ca57f1-469d-11e7-af02-69e470af7417\",\"color\":\"#68BC00\",\"split_mode\":\"everything\",\"metrics\":[{\"id\":\"61ca57f2-469d-11e7-af02-69e470af7417\",\"type\":\"max\",\"field\":\"node_stats.os.cpu.load_average.1m\"},{\"script\":\"params.foo > 0 ? params.foo : 0\",\"id\":\"c21b4720-e2ac-11e8-a9a4-c1de06a90990\",\"type\":\"calculation\",\"variables\":[{\"id\":\"c521d060-e2ac-11e8-a9a4-c1de06a90990\",\"name\":\"foo\",\"field\":\"61ca57f2-469d-11e7-af02-69e470af7417\"}]}],\"separate_axis\":0,\"axis_position\":\"right\",\"formatter\":\"number\",\"chart_type\":\"line\",\"line_width\":1,\"point_size\":1,\"fill\":0.5,\"stacked\":\"none\"}],\"time_field\":\"timestamp\",\"index_pattern\":\".monitoring-es*\",\
@jakelandis
jakelandis / gist:86ac7e2544f96ded3f076bfedd54a02c
Created November 13, 2018 18:49
./gradlew :x-pack:qa:smoke-test-watcher-with-security:integTestRunner -Dtests.seed=1FEC4B295D693CFE -Dtests.class=org.elasticsearch.smoketest.SmokeTestWatcherWithSecurityIT -Dtests.method="testSearchInputWithInsufficientPrivileges" -Dtests.security.manager=true -Dtests.locale=hi-IN -Dtests.timezone=Asia/Sakhalin -Dcompiler.java=11 -Druntime.java=8
23:50:07 > Task :x-pack:qa:smoke-test-watcher-with-security:integTestCluster#wait
23:50:07 Task ':x-pack:qa:smoke-test-watcher-with-security:integTestCluster#wait' is not up-to-date because:
23:50:07 Task has not declared any outputs despite executing actions.
23:50:17 [ant:get] Getting: http://[::1]:60232/_cluster/health?wait_for_nodes=>=1&wait_for_status=yellow
23:50:17 [get] Getting: http://[::1]:60232/_cluster/health?wait_for_nodes=>=1&wait_for_status=yellow
23:50:17 [ant:get] To: /private/var/lib/jenkins/workspace/elastic+elasticsearch+master+multijob-darwin-compatibility/x-pack/qa/smoke-test-watcher-with-security/build/cluster/integTestCluster node0/
@jakelandis
jakelandis / gist:9ef72d6bff8a7e3d33922a9cde0c4063
Created June 14, 2019 19:30
stack-over-flow
[2019-05-13T09:07:53,843][WARN ][o.e.x.c.a.AutoFollowCoordinator] [hostname.was.here] failure occurred while fetching cluster state for auto follow pattern [follow-pattern-was-here]
org.elasticsearch.ElasticsearchStatusException: can not fetch remote cluster state as the license state of the remote cluster [remote-cluster-name-was-here] could not be determined
at org.elasticsearch.xpack.ccr.CcrLicenseChecker.clusterStateUnknownRemoteLicense(CcrLicenseChecker.java:442) ~[?:?]
at org.elasticsearch.xpack.ccr.CcrLicenseChecker.lambda$checkRemoteClusterLicenseAndFetchClusterState$5(CcrLicenseChecker.java:181) ~[?:?]
at org.elasticsearch.xpack.ccr.CcrLicenseChecker$1.onFailure(CcrLicenseChecker.java:232) ~[?:?]
at org.elasticsearch.license.RemoteClusterLicenseChecker$1.onFailure(RemoteClusterLicenseChecker.java:188) ~[?:?]
at org.elasticsearch.action.support.ContextPreservingActionListener.onFailure(ContextPreservingActionListener.java:50) ~[elasticsearch-6.7.1.jar:6.7.1]
at org.elasticsearch.action.ActionLis
@jakelandis
jakelandis / test.json
Created July 24, 2019 21:39
{
"$schema": "http://json-schema.org/draft-07/schema#",
"type": "object",
"properties": {
"one": {
"type": "string"
},
"two": {
"type": "string"
},