Alex-Ikanow

DBCollection.prototype.getSplitKeysForChunks = function ( keyPattern, chunkSize ){

   var stats = this.stats()

   if( ! stats.sharded ){
       print( "Collection " + this + " is not sharded." )
       return
   }

   var config = this.getMongo().getDB("config")

Alex-Ikanow

#### 1] Install works fine....

[acp@localhost elasticsearch]$ sudo bin/plugin -install lukas-vlcek/bigdesk
-> Installing lukas-vlcek/bigdesk...
Trying https://github.com/downloads/lukas-vlcek/bigdesk/bigdesk-0.17.9.zip...
Trying https://github.com/lukas-vlcek/bigdesk/zipball/v0.17.9...
Trying https://github.com/lukas-vlcek/bigdesk/zipball/master...
Downloading ..................................DONE
Identified as a _site plugin, moving to _site structure ...
Installed bigdesk

Alex-Ikanow

curl -XGET 'http://betadev:9200/document_index,doc_COMMUNITYID/document_index' -d '
{
  "query":{
    "bool":{
      "must":[{
          "bool":{
            "must":{
              "match_all":{}
            }
          }

Alex-Ikanow

I managed to get the query to fail a few different ways, one of them appears to be linked to my "random" use of "nested" as the index type while trying to reproduce the "operational" error I occurred.

1] Create an index called "nested_test" (eg I did this from elasticsearch-head) with 5 shards and 0 replicas

(or presumably something like 
curl -XPUT 'http://localhost:9200/nested_test/' -d '
index :
    number_of_shards : 5
    number_of_replicas : 0

Alex-Ikanow

Steps 1-3 are the same as part 1 (https://gist.github.com/1453137)

1] Create an index called "nested_test" (eg I did this from elasticsearch-head) with 5 shards and 0 replicas

(or presumably something like 
curl -XPUT 'http://localhost:9200/nested_test/' -d '
index :
    number_of_shards : 5
    number_of_replicas : 0
'

Alex-Ikanow

function s1(el) {
	if (el instanceof Array) {
		s2(el, 1);
	}
	else if (typeof el == 'object') {
		outList.add(s3(el));
	}
	else {
		outList.add(el);
	}

Alex-Ikanow

// First document

{
  "_id": "4f2723ae4f3625d892f5faf4",
  "title": "Profile 1",
  // Lots of other completely mapped fields, including arrays of objects, geo points, strings etc
  "metadata": { // metadata does not have a mapping
    "socialnetworkobject": [
      {
        "field1": {}, // <-- THIS CAUSES THE PROBLEM WHEN A DOC WITH A STRING VALUE HERE IS INSERTED

Alex-Ikanow

// Unfortunately this attempt at a simple recreation doesn't work.
// This differences vs the real world case (that occurs 100% consistently are):
//  - the "obj1" equivalent is not a completely dynamic object, all fields are declared via mapping apart from obj2
//  - the "obj1" mapping has a long and complex set of dynamic templates declared (none of which are used)
//  - many other documents not containing any obj2 fields have been indexed before the first value with a "bad" obj2 is indexed
// (- several GBs of other shards/indexes etc etc are present)

// 1] Create an index, eg something like 

Alex-Ikanow

2012-02-08 17:50:41.252 [DEBUG] action.search.type:89 - [Screaming Mimi] [doc_dummy][0], node[x1GaMz1BQBSIzbcmc5GoTQ], [P], s[STARTED]: Failed to execute [org.elasticsearch.action.search.SearchRequest@5fdb7adc]
org.elasticsearch.search.SearchParseException: [doc_dummy][0]: from[-1],size[1000]: Parse Failure [Failed to parse source [:)
?ze$??query?ol?st?ustom_score??ould?sted?term?dexXtest systems/industryterm?athGentities?score_modeBmax??scriptJdecayscript?langEnative?params?ram?x)?x)?x)=} c=YIh;%Weu?"??nstant_score?lter??rms?mmunityId?927585d591d31d7b37097aW4c927585d591d31d7c37097bW4e53e755ebcd573d6af5f091W4e6e1deaacd7852244b92fbfW4da6080df891e8e224809c41W4e08f20f0e9a573dd8f74b93??ost(??ript_fields?cay?ecayscriptLEnativeM??x)?x)?x)=} c=YIh;%Weu?#??facets?o?fieldClocs@$??ime?te_histogram?ublishedDate?intervalA1w?vents?Jassoc_index@$??regexI^Event\|.*?ssociations?cts?Jassoc_index@$?]H^Fact\|.*?ssociations?urceTags?Ctags@$??ourceTypes?HmediaType@$??ourceKeys?HsourceKey@$??]]
        at org.elasticsearch.search.

Alex-Ikanow

umount /mnt
mdadm --create /dev/md0 --run --level=0 --chunk=256 --raid-devices=4 /dev/sdb /dev/sdc /dev/sdd /dev/sde
pvcreate /dev/md0
vgcreate data_vg /dev/md0
lvcreate -l 100%vg -n data_vol data_vg
mkfs.ext3 -m 0 /dev/data_vg/data_vol
mount -t ext3 /dev/data_vg/data_vol /mnt/

touch /etc/mdadm.conf
grep -F "/dev/md0" /etc/mdadm.conf || mdadm -Es | grep md0 >> /etc/mdadm.conf