Created
March 8, 2013 10:30
-
-
Save BlueStalker/5115578 to your computer and use it in GitHub Desktop.
Terms Facets with few hits got Elastic Out Of Memory Error.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Java Version: jdk1.7.0_05 | |
ElasticSearch Engine Version: elasticsearch-0.20.5 | |
Cluster Info: We have one master node, and 2 slave nodes, with all : | |
ES_HEAP_SIZE=3000M | |
ES_MIN_MEM=3000M | |
ES_MAX_MEM=3000M | |
On Master Node: ps aux | grep elas | |
root 1447 9.6 45.0 3840652 3679364 ? SLl Mar07 138:52 /root/jdk1.7.0_05/bin/java -Xms3000M -Xmx3000M -Xss256k -Djava.awt.headless=true -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError -Delasticsearch -Des.path.home=/root/elasticsearch-0.20.5 -cp :/root/elasticsearch-0.20.5/lib/elasticsearch-0.20.5.jar:/root/elasticsearch-0.20.5/lib/*:/root/elasticsearch-0.20.5/lib/sigar/* org.elasticsearch.bootstrap.ElasticSearch | |
Index name: surfikisemterms | |
the mapping is: | |
{"surfikisemterms":{"jdbc_search":{"properties":{"facets":{"dynamic":"true","properties":{"termscount":{"dynamic":"true","properties":{"terms":{"dynamic":"true","properties":{"field":{"type":"string"},"size":{"type":"long"}}}}}}},"from":{"type":"long"},"query":{"dynamic":"true","properties":{"bool":{"dynamic":"true","properties":{"must":{"dynamic":"true","properties":{"query_string":{"dynamic":"true","properties":{"query":{"type":"string"}}}}}}}}},"size":{"type":"long"}}},"jdbc":{"properties":{"keywords":{"type":"string","analyzer":"comma"},"time":{"type":"date","format":"dateOptionalTime"}}}}} | |
index setting is: | |
{"surfikisemterms":{"settings":{"index.analysis.tokenizer.commatokenizer.type":"pattern","index.analysis.analyzer.comma.type":"custom","index.number_of_replicas":"1","index.version.created":"200599","index.analysis.tokenizer.commatokenizer.pattern":",","index.analysis.analyzer.comma.tokenizer":"commatokenizer","index.number_of_shards":"5"}}} | |
So actually we have 2 fields in the surfikisemterms, | |
1. keywords, a list of string | |
2. time, the timestamp for these keywords | |
The query: | |
{ | |
"query": { | |
"bool": { | |
"must": [ | |
{ | |
"range": { | |
"time": { | |
"from": "2013-03-07T11:29:19.000Z", | |
"to": "2013-03-07T11:30:19.000Z" | |
} | |
} | |
} | |
], | |
"must_not": [], | |
"should": [] | |
} | |
}, | |
"from": 0, | |
"size": 10, | |
"sort": [], | |
"facets": {} | |
} | |
Above query will return about 20 hits, with each hit the keyword list will no more than 20. | |
So, the terms facets according to above query: | |
Then if I do: | |
{ | |
"query": { | |
"bool": { | |
"must": [ | |
{ | |
"range": { | |
"time": { | |
"from": "2013-03-07T11:29:19.000Z", | |
"to": "2013-03-07T11:30:19.000Z" | |
} | |
} | |
} | |
], | |
"must_not": [], | |
"should": [] | |
} | |
}, | |
"from": 0, | |
"size": 22, | |
"sort": [], | |
"facets": { | |
"termscount": { | |
"terms": { | |
"field": "keywords", | |
"size": 25 | |
} | |
} | |
} | |
} | |
I can see following logs (on master node, name is Bobby Hutcherson) repeating for several time: | |
[2013-03-08 03:31:05,571][WARN ][index.cache.field.data.resident] [Surfiki | |
Master: Bobby Hutcherson] [surfikisemterms] loading field [keywords] caused | |
out of memory failure | |
java.lang.OutOfMemoryError: Java heap space.. | |
Thanks. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment