Create a gist now

Instantly share code, notes, and snippets.

What would you like to do?
Configuring Tire to work with Bonsai

1. Configure Tire to use the Bonsai ElasticSearch Heroku add-on

gem 'tire'

config/initializers/bonsai.rb

ENV['ELASTICSEARCH_URL'] = ENV['BONSAI_URL']

app/models/article.rb

class Article
  include Tire::Model::Search
  include Tire::Model::Callbacks
end

2. Create the index and import your documents

rake environment tire:import CLASS=Article FORCE=true

Known issues

There are no "known issues" with Tire and Bonsai at the moment. Having trouble with something in particular? Drop us a line at support@bonsai.io.

  • Custom index analyzers must be set at index creation time. Ability to dynamically create, modify and destroy indexes.
  • Bulk import uses cluster-level /_bulk handler rather than the index-level _bulk handler, causing bulk imports to fail. Issue 327
  • Multi-model search is not scoped within the index. Issue 322
  • Undefined method [] for nil:NilClass on index.settings. Bonsai shared cluster indices are mapped to a random identifier, whereas Tire expects the logical index name in the _index response. To be fixed within Bonsai. Email support@bonsai.io if this is affecting you. Issue 386
  • ES Alias API not fully supported in Bonsai. Email support@bonsai.io if you would like to beta test that.
@ernesto-jimenez

This comment has been minimized.

Show comment Hide comment
@ernesto-jimenez

ernesto-jimenez Mar 18, 2012

Tire.configuration doesn't seem to work in Tire's latest version.

I've used:

Tire.configure do
  url "http://index.bonsai.io"
end

Tire.configuration doesn't seem to work in Tire's latest version.

I've used:

Tire.configure do
  url "http://index.bonsai.io"
end
@nz

This comment has been minimized.

Show comment Hide comment
@nz

nz Mar 19, 2012

Owner

nz commented Mar 19, 2012

@stanBienaives

This comment has been minimized.

Show comment Hide comment
@stanBienaives

stanBienaives Mar 31, 2012

Hey,
I got a "undefined method `URL' for main:Object" . does one of you have a hint?

thanks

Hey,
I got a "undefined method `URL' for main:Object" . does one of you have a hint?

thanks

@ernesto-jimenez

This comment has been minimized.

Show comment Hide comment
@ernesto-jimenez

ernesto-jimenez Mar 31, 2012

@kikouli, use

 URI.parse(ENV['BONSAI_INDEX_URL']).path[1..-1]

@kikouli, use

 URI.parse(ENV['BONSAI_INDEX_URL']).path[1..-1]
@stanBienaives

This comment has been minimized.

Show comment Hide comment
@stanBienaives

stanBienaives Apr 1, 2012

thanks a lot ernesto . it works now!

thanks a lot ernesto . it works now!

@jayinteractive

This comment has been minimized.

Show comment Hide comment
@jayinteractive

jayinteractive Apr 4, 2012

Silly question but this now fails in my development environment throwing the obvious error

ruby-1.9.2-p318/lib/ruby/1.9.1/uri/common.rb:156:in `split': bad URI(is not URI?):  (URI::InvalidURIError)

What's the fix to getting development back please

Silly question but this now fails in my development environment throwing the obvious error

ruby-1.9.2-p318/lib/ruby/1.9.1/uri/common.rb:156:in `split': bad URI(is not URI?):  (URI::InvalidURIError)

What's the fix to getting development back please

@ernesto-jimenez

This comment has been minimized.

Show comment Hide comment
@ernesto-jimenez

ernesto-jimenez Apr 4, 2012

Hi @jayinteractive

This configuration uses the BONSAI_INDEX_URL environment variable set up the bonsai add-on in heroku.

You'll need to set the variable in your development environment with the URL to your development ElasticSearch index.

In my case, rather than setting up another environment variable, I installed ElasticSearch in my development machine and modified the initializer:

if ENV['BONSAI_INDEX_URL']
  bonsai_uri = URI.parse(ENV['BONSAI_INDEX_URL'])
  Tire.configure do
    url "http://index.bonsai.io"
  end
  BONSAI_INDEX_NAME = bonsai_uri.path[1..-1]
else
  BONSAI_INDEX_NAME = "my_index"
end

Hi @jayinteractive

This configuration uses the BONSAI_INDEX_URL environment variable set up the bonsai add-on in heroku.

You'll need to set the variable in your development environment with the URL to your development ElasticSearch index.

In my case, rather than setting up another environment variable, I installed ElasticSearch in my development machine and modified the initializer:

if ENV['BONSAI_INDEX_URL']
  bonsai_uri = URI.parse(ENV['BONSAI_INDEX_URL'])
  Tire.configure do
    url "http://index.bonsai.io"
  end
  BONSAI_INDEX_NAME = bonsai_uri.path[1..-1]
else
  BONSAI_INDEX_NAME = "my_index"
end
@nz

This comment has been minimized.

Show comment Hide comment
@nz

nz Apr 6, 2012

Thanks for all the discussion here, folks. Open source is awesome. Keep it up!

Updated the gist to drop the URI.parse in favor of String#[Regexp] based on the issues mentioned above with it.

Also made BONSAI_INDEX_NAME conditional on the presence of ENV['BONSAI_INDEX_URL'] as per @ernesto-jimenez's comment. My version sets its value to the application name and environment. So your index for AcmeBlog in development would be acme-blog-development.

Owner

nz commented Apr 6, 2012

Thanks for all the discussion here, folks. Open source is awesome. Keep it up!

Updated the gist to drop the URI.parse in favor of String#[Regexp] based on the issues mentioned above with it.

Also made BONSAI_INDEX_NAME conditional on the presence of ENV['BONSAI_INDEX_URL'] as per @ernesto-jimenez's comment. My version sets its value to the application name and environment. So your index for AcmeBlog in development would be acme-blog-development.

@rahilsondhi

This comment has been minimized.

Show comment Hide comment
@rahilsondhi

rahilsondhi Apr 21, 2012

I tried following these instructions and I'm getting this in the Heroku logs:

2012-04-21T23:18:00+00:00 app[web.1]: Started GET "/search.json?q=janice" for 67.224.81.78 at 2012-04-21 19:18:00 -0400
2012-04-21T23:18:01+00:00 app[web.1]: Processing by SearchController#index as JSON
2012-04-21T23:18:01+00:00 app[web.1]:   Parameters: {"q"=>"janice"}
2012-04-21T23:18:01+00:00 app[web.1]: [REQUEST FAILED] curl -X GET "http://index.bonsai.io/artists,users/_search?pretty=true" -d '{"query":{"query_string":{"query":"janice"}}}'
2012-04-21T23:18:01+00:00 app[web.1]: Completed 500 Internal Server Error in 25ms
2012-04-21T23:18:01+00:00 app[web.1]: 
2012-04-21T23:18:01+00:00 app[web.1]: Tire::Search::SearchRequestFailed (401 : {"error": "Not authorized: Some endpoints are admin-only, ask support@onemorecloud.com."}
2012-04-21T23:18:01+00:00 app[web.1]: ):
2012-04-21T23:18:01+00:00 app[web.1]:   app/models/user.rb:141:in `search_for'
2012-04-21T23:18:01+00:00 app[web.1]:   app/controllers/search_controller.rb:6:in `index'

I tried following these instructions and I'm getting this in the Heroku logs:

2012-04-21T23:18:00+00:00 app[web.1]: Started GET "/search.json?q=janice" for 67.224.81.78 at 2012-04-21 19:18:00 -0400
2012-04-21T23:18:01+00:00 app[web.1]: Processing by SearchController#index as JSON
2012-04-21T23:18:01+00:00 app[web.1]:   Parameters: {"q"=>"janice"}
2012-04-21T23:18:01+00:00 app[web.1]: [REQUEST FAILED] curl -X GET "http://index.bonsai.io/artists,users/_search?pretty=true" -d '{"query":{"query_string":{"query":"janice"}}}'
2012-04-21T23:18:01+00:00 app[web.1]: Completed 500 Internal Server Error in 25ms
2012-04-21T23:18:01+00:00 app[web.1]: 
2012-04-21T23:18:01+00:00 app[web.1]: Tire::Search::SearchRequestFailed (401 : {"error": "Not authorized: Some endpoints are admin-only, ask support@onemorecloud.com."}
2012-04-21T23:18:01+00:00 app[web.1]: ):
2012-04-21T23:18:01+00:00 app[web.1]:   app/models/user.rb:141:in `search_for'
2012-04-21T23:18:01+00:00 app[web.1]:   app/controllers/search_controller.rb:6:in `index'
@nz

This comment has been minimized.

Show comment Hide comment
@nz

nz Apr 22, 2012

@rahilsondhi, your URL there (…index.bonsai.io/artists,users…) is omitting your index name, which is the random 20-digit base-36 string from the BONSAI_INDEX_URL environment variable. That's turning your search into a multi-index search across two indexes that don't exist. (Which we also wouldn't allow at this point, since no app should need to search across multiple indexes anyway.)

Another cluster-level Tire method call that @karmi should know about?

Owner

nz commented Apr 22, 2012

@rahilsondhi, your URL there (…index.bonsai.io/artists,users…) is omitting your index name, which is the random 20-digit base-36 string from the BONSAI_INDEX_URL environment variable. That's turning your search into a multi-index search across two indexes that don't exist. (Which we also wouldn't allow at this point, since no app should need to search across multiple indexes anyway.)

Another cluster-level Tire method call that @karmi should know about?

@rahilsondhi

This comment has been minimized.

Show comment Hide comment
@rahilsondhi

rahilsondhi Apr 23, 2012

So what do you recommend I do @nz? My initializer and models are set up like the instructions ask for.

So what do you recommend I do @nz? My initializer and models are set up like the instructions ask for.

@nz

This comment has been minimized.

Show comment Hide comment
@nz

nz Apr 23, 2012

Owner

nz commented Apr 23, 2012

@nz

This comment has been minimized.

Show comment Hide comment
@nz

nz Apr 30, 2012

Thanks for the continued feedback folks. I've updated the gist with some Tire known problems, and their Issues on GitHub:

Owner

nz commented Apr 30, 2012

Thanks for the continued feedback folks. I've updated the gist with some Tire known problems, and their Issues on GitHub:

@nz

This comment has been minimized.

Show comment Hide comment
@nz

nz Apr 30, 2012

Fix for bulk importing awaiting testing, merge and release: karmi/tire#328 — feedback welcome!

Owner

nz commented Apr 30, 2012

Fix for bulk importing awaiting testing, merge and release: karmi/tire#328 — feedback welcome!

@nz

This comment has been minimized.

Show comment Hide comment
@nz

nz May 3, 2012

Hey all, Tire 0.4.1 was just shipped with some fixes for bulk imports. Upgrade and import and drop me a line here or info@onemorecloud.com with any feedback.

Owner

nz commented May 3, 2012

Hey all, Tire 0.4.1 was just shipped with some fixes for bulk imports. Upgrade and import and drop me a line here or info@onemorecloud.com with any feedback.

@zacksiri

This comment has been minimized.

Show comment Hide comment
@zacksiri

zacksiri Jun 24, 2012

there is a problem with this technique

if your setting the index_name in the class you will run into the problem that the Callbacks are not working since the instance.index != Class.index

the work around for this is to create your own observer for after_save and after_destroy

def after_save(record)
   Class.index.store(record)
end

def after_destroy(record)
   Class.index.remove(record)
end

since I only have 1 model i am using this works for me but it can be pretty tedious if you have multiple model maybe you can use active record observer that is not inferred

read more about it here. http://api.rubyonrails.org/classes/ActiveRecord/Observer.html

there is a problem with this technique

if your setting the index_name in the class you will run into the problem that the Callbacks are not working since the instance.index != Class.index

the work around for this is to create your own observer for after_save and after_destroy

def after_save(record)
   Class.index.store(record)
end

def after_destroy(record)
   Class.index.remove(record)
end

since I only have 1 model i am using this works for me but it can be pretty tedious if you have multiple model maybe you can use active record observer that is not inferred

read more about it here. http://api.rubyonrails.org/classes/ActiveRecord/Observer.html

@verdi327

This comment has been minimized.

Show comment Hide comment
@verdi327

verdi327 Jul 4, 2012

so, I followed instructions in the gist, adding this to my initializers/bonsai.rb

if ENV['BONSAI_INDEX_URL'] Tire.configure do url "http://index.bonsai.io" end BONSAI_INDEX_NAME = ENV['BONSAI_INDEX_URL'][/[^\/]+$/] else app_name = Rails.application.class.parent_name.underscore.dasherize app_env = Rails.env BONSAI_INDEX_NAME = "#{app_name}-#{app_env}" end

Also added this to the model

index_name BONSAI_INDEX_NAME

Now, when I run my app locally in development I get the following error

[REQUEST FAILED] curl -X GET "http://localhost:9200/dynamite-urbanite-development/city/_search?load=true&pretty=true" -d '{"query":{"query_string":{"query":"texas","default_operator":"AND"}}}'

Started GET "/cities?utf8=%E2%9C%93&query=texas" for 127.0.0.1 at 2012-07-03 22:12:02 -0400
Connecting to database specified by database.yml
Processing by CitiesController#index as HTML
Parameters: {"utf8"=>"✓", "query"=>"texas"}
Completed 500 Internal Server Error in 3ms

Tire::Search::SearchRequestFailed (404 : {"error":"IndexMissingException[[dynamite-urbanite-development] missing]","status":404}):
app/models/city.rb:20:in search' app/controllers/cities_controller.rb:3:inindex'

verdi327 commented Jul 4, 2012

so, I followed instructions in the gist, adding this to my initializers/bonsai.rb

if ENV['BONSAI_INDEX_URL'] Tire.configure do url "http://index.bonsai.io" end BONSAI_INDEX_NAME = ENV['BONSAI_INDEX_URL'][/[^\/]+$/] else app_name = Rails.application.class.parent_name.underscore.dasherize app_env = Rails.env BONSAI_INDEX_NAME = "#{app_name}-#{app_env}" end

Also added this to the model

index_name BONSAI_INDEX_NAME

Now, when I run my app locally in development I get the following error

[REQUEST FAILED] curl -X GET "http://localhost:9200/dynamite-urbanite-development/city/_search?load=true&pretty=true" -d '{"query":{"query_string":{"query":"texas","default_operator":"AND"}}}'

Started GET "/cities?utf8=%E2%9C%93&query=texas" for 127.0.0.1 at 2012-07-03 22:12:02 -0400
Connecting to database specified by database.yml
Processing by CitiesController#index as HTML
Parameters: {"utf8"=>"✓", "query"=>"texas"}
Completed 500 Internal Server Error in 3ms

Tire::Search::SearchRequestFailed (404 : {"error":"IndexMissingException[[dynamite-urbanite-development] missing]","status":404}):
app/models/city.rb:20:in search' app/controllers/cities_controller.rb:3:inindex'

@verdi327

This comment has been minimized.

Show comment Hide comment
@verdi327

verdi327 Jul 4, 2012

So, I fixed the above problem by re - indexing my City model. My app works in development, but when I pushed the changes to Heroku, the application now crashes. Any ideas? The error message from Heroku logs are not help enough, all they tell me is that my app did indeed crash.

verdi327 commented Jul 4, 2012

So, I fixed the above problem by re - indexing my City model. My app works in development, but when I pushed the changes to Heroku, the application now crashes. Any ideas? The error message from Heroku logs are not help enough, all they tell me is that my app did indeed crash.

@pawel2105

This comment has been minimized.

Show comment Hide comment
@pawel2105

pawel2105 Sep 28, 2012

Have setup the initializer as above and when I curl the URL for BONSAI_INDEX_URL found in heroku config the test response is successful but when interacting with a model I get the infamous connection refused error. Is there a bug with Tire 0.4.2 or something?

Have setup the initializer as above and when I curl the URL for BONSAI_INDEX_URL found in heroku config the test response is successful but when interacting with a model I get the infamous connection refused error. Is there a bug with Tire 0.4.2 or something?

@pawel2105

This comment has been minimized.

Show comment Hide comment
@pawel2105

pawel2105 Oct 1, 2012

bump

bump

@pawel2105

This comment has been minimized.

Show comment Hide comment
@pawel2105

pawel2105 Oct 1, 2012

For debugging:

Trying to curl locally works, but trying to curl from heroku returns the following:

heroku run curl -XPOST http://index.bonsai.io/e2ljko3c8qhc53zmdf0/test/hello -d '{"title":"Hello world"}' gives this...

Runningcurl -XPOST http://index.bonsai.io/e2ljko3c8qhc53zmdf0/test/hello -d {"title":"Hello world"}attached to terminal... up, run.1 {"error":"RemoteTransportException[[Polaris][inet[/10.31.156.223:9300]][index]]; nested: MapperParsingException[Failed to parse]; nested: JsonParseException[Unexpected character ('H' (code 72)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')\n at [Source: [B@1f8fbddc; line: 1, column: 117]]; ","status":400}

For debugging:

Trying to curl locally works, but trying to curl from heroku returns the following:

heroku run curl -XPOST http://index.bonsai.io/e2ljko3c8qhc53zmdf0/test/hello -d '{"title":"Hello world"}' gives this...

Runningcurl -XPOST http://index.bonsai.io/e2ljko3c8qhc53zmdf0/test/hello -d {"title":"Hello world"}attached to terminal... up, run.1 {"error":"RemoteTransportException[[Polaris][inet[/10.31.156.223:9300]][index]]; nested: MapperParsingException[Failed to parse]; nested: JsonParseException[Unexpected character ('H' (code 72)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')\n at [Source: [B@1f8fbddc; line: 1, column: 117]]; ","status":400}

@bryanmtl

This comment has been minimized.

Show comment Hide comment
@bryanmtl

bryanmtl Oct 17, 2012

Using Tire 0.4.3, I'm receiving the following when trying to bulk import records:

<Tire::Index:0x00000008d5a770 @name="myindexname", @response=502 :

502 Bad Gateway

Using Tire 0.4.3, I'm receiving the following when trying to bulk import records:

<Tire::Index:0x00000008d5a770 @name="myindexname", @response=502 :

502 Bad Gateway

@nz

This comment has been minimized.

Show comment Hide comment
@nz

nz Oct 18, 2012

For those interested, I just soft deployed a big change to how provisioning works, which should be much more compatible with how Tire operates.

New addons will get a different environment variable: BONSAI_URL, which can be set directly for the Tire URL. You can now create indexes at this URL using the usual ElasticSearch Index APIs. Specifying the index name is now optional, though recommended, since I still think that applications should only need a single index per environment, as with your database.

I'll announce more about that on Monday, in particular about number of indexes and shards allowed within our production plans. In the mean time, check out the revised gist, and let me know if you have any questions. You can always email me directly at nick@onemorecloud.com as well.

Cheers, folks, and thanks for all the great feedback throughout the beta! Looking forward to launching this service very soon :-)

/cc @bryanmtl @pawel2105 @verdi327 @zacksiri @rahilsondhi @ernesto-jimenez @jayinteractive @kikouli

Owner

nz commented Oct 18, 2012

For those interested, I just soft deployed a big change to how provisioning works, which should be much more compatible with how Tire operates.

New addons will get a different environment variable: BONSAI_URL, which can be set directly for the Tire URL. You can now create indexes at this URL using the usual ElasticSearch Index APIs. Specifying the index name is now optional, though recommended, since I still think that applications should only need a single index per environment, as with your database.

I'll announce more about that on Monday, in particular about number of indexes and shards allowed within our production plans. In the mean time, check out the revised gist, and let me know if you have any questions. You can always email me directly at nick@onemorecloud.com as well.

Cheers, folks, and thanks for all the great feedback throughout the beta! Looking forward to launching this service very soon :-)

/cc @bryanmtl @pawel2105 @verdi327 @zacksiri @rahilsondhi @ernesto-jimenez @jayinteractive @kikouli

@pietia

This comment has been minimized.

Show comment Hide comment
@pietia

pietia Oct 21, 2012

Can't wait to see updated notes :)

pietia commented Oct 21, 2012

Can't wait to see updated notes :)

@pietia

This comment has been minimized.

Show comment Hide comment
@pietia

pietia Oct 22, 2012

@nz, btw do you use latest Tire version from github or just latest stable release?

pietia commented Oct 22, 2012

@nz, btw do you use latest Tire version from github or just latest stable release?

@zefer

This comment has been minimized.

Show comment Hide comment
@zefer

zefer Oct 31, 2012

@nz is it possible to set the default analyzer on the _all field? I have tried this with the following settings in an ActiveModel

settings index: { analysis: { analyzer: { default: { type: 'snowball', language: 'english' } } } }

...but it looks like Bonsai is silently rejecting them...

curl -XGET 'http://juniper-1234.us-east-1.bonsai.io/artworks_staging/_settings'                             
{"xxxx1234":{"settings":{"index.number_of_replicas":"1","index.number_of_shards":"1","index.version.created":"190999"}}}

I need to find a way to have the _all field analyzed with snowball. Do you/anyone know a solution to this?

zefer commented Oct 31, 2012

@nz is it possible to set the default analyzer on the _all field? I have tried this with the following settings in an ActiveModel

settings index: { analysis: { analyzer: { default: { type: 'snowball', language: 'english' } } } }

...but it looks like Bonsai is silently rejecting them...

curl -XGET 'http://juniper-1234.us-east-1.bonsai.io/artworks_staging/_settings'                             
{"xxxx1234":{"settings":{"index.number_of_replicas":"1","index.number_of_shards":"1","index.version.created":"190999"}}}

I need to find a way to have the _all field analyzed with snowball. Do you/anyone know a solution to this?

@digitalplaywright

This comment has been minimized.

Show comment Hide comment
@digitalplaywright

digitalplaywright Oct 31, 2012

The instructions on

https://devcenter.heroku.com/articles/bonsai#using-tire-with-rails-3x 

does not mention setting up a custom index 'index_name INDEX_NAME' for every model. Are the instructions in this gist still necessary for the current version of Bonsai Search?

The instructions on

https://devcenter.heroku.com/articles/bonsai#using-tire-with-rails-3x 

does not mention setting up a custom index 'index_name INDEX_NAME' for every model. Are the instructions in this gist still necessary for the current version of Bonsai Search?

@nz

This comment has been minimized.

Show comment Hide comment
@nz

nz Nov 1, 2012

@digitalplaywright: the "shared index" approach — specifying the same index_name per model — is now optional. You may now create multiple indexes, subject to the total shards allowed per plan. (Currently set to two during beta.)

Owner

nz commented Nov 1, 2012

@digitalplaywright: the "shared index" approach — specifying the same index_name per model — is now optional. You may now create multiple indexes, subject to the total shards allowed per plan. (Currently set to two during beta.)

@nz

This comment has been minimized.

Show comment Hide comment
@nz

nz Nov 1, 2012

@pietia any relatively recent version of Tire should be fine, I'm working to support what's popular

Owner

nz commented Nov 1, 2012

@pietia any relatively recent version of Tire should be fine, I'm working to support what's popular

@nz

This comment has been minimized.

Show comment Hide comment
@nz

nz Nov 1, 2012

@zefer can you drop me an email (support@onemorecloud.com) with the full index URL, and perhaps a curl command that's trying to POST or PUT the settings? We do filter that settings payload to track/enforce shards and replicas, but analyzers should pass through as provided.

Owner

nz commented Nov 1, 2012

@zefer can you drop me an email (support@onemorecloud.com) with the full index URL, and perhaps a curl command that's trying to POST or PUT the settings? We do filter that settings payload to track/enforce shards and replicas, but analyzers should pass through as provided.

@evanwhalen

This comment has been minimized.

Show comment Hide comment
@evanwhalen

evanwhalen Jan 14, 2013

Adding the following to an initializer will set the index prefix for all models:

Tire::Model::Search.index_prefix "#{Rails.application.class.parent_name.downcase}_#{Rails.env.to_s.downcase}"

Adding the following to an initializer will set the index prefix for all models:

Tire::Model::Search.index_prefix "#{Rails.application.class.parent_name.downcase}_#{Rails.env.to_s.downcase}"
@nz

This comment has been minimized.

Show comment Hide comment
@nz

nz Jan 14, 2013

Nice tip, @evanwhalen. I wonder if something like that could be suggested for a contribution upstream to Tire proper. Scoping by application and environment in some way or another seems like a no-brainer for local development and testing with multiple applications.

Owner

nz commented Jan 14, 2013

Nice tip, @evanwhalen. I wonder if something like that could be suggested for a contribution upstream to Tire proper. Scoping by application and environment in some way or another seems like a no-brainer for local development and testing with multiple applications.

@miry

This comment has been minimized.

Show comment Hide comment
@miry

miry Jan 23, 2013

I still have trouble with search with multiple indexes.

> Tire.search("profiles,companies") { query { string "*world*"} }

[REQUEST FAILED] curl -X GET 'http://username@juniper-2764614.us-east-1.bonsai.io/profiles,companies/_search?pretty' -d '{"query":{"query_string":{"query":"*world*"}}}'
Tire::Search::SearchRequestFailed: 404 : {"error": "Index not found", "status": 404}

Is it possible to use Bonsai with multiple indexes search?

miry commented Jan 23, 2013

I still have trouble with search with multiple indexes.

> Tire.search("profiles,companies") { query { string "*world*"} }

[REQUEST FAILED] curl -X GET 'http://username@juniper-2764614.us-east-1.bonsai.io/profiles,companies/_search?pretty' -d '{"query":{"query_string":{"query":"*world*"}}}'
Tire::Search::SearchRequestFailed: 404 : {"error": "Index not found", "status": 404}

Is it possible to use Bonsai with multiple indexes search?

@Will-Sommers

This comment has been minimized.

Show comment Hide comment
@Will-Sommers

Will-Sommers Feb 7, 2013

miry, I have the same problem as well. I'm looking into creating an index that the merged version of the index I'd like to search.

miry, I have the same problem as well. I'm looking into creating an index that the merged version of the index I'd like to search.

@nz

This comment has been minimized.

Show comment Hide comment
@nz

nz Feb 7, 2013

@miry and @Will-Sommers, this issue is on my end. I still have some work to do for supporting multi-index search syntax in Elasticsearch. I'll take a stab at that this week.

Owner

nz commented Feb 7, 2013

@miry and @Will-Sommers, this issue is on my end. I still have some work to do for supporting multi-index search syntax in Elasticsearch. I'll take a stab at that this week.

@nz

This comment has been minimized.

Show comment Hide comment
@nz

nz Feb 7, 2013

@miry @Will-Sommers — in the mean time, merging into a combined index (as per the "optional" in the instructions) should work fine for your purposes. You can then execute searches with:

Tire.search(INDEX_NAME) { query { string 'hello world' } }
Owner

nz commented Feb 7, 2013

@miry @Will-Sommers — in the mean time, merging into a combined index (as per the "optional" in the instructions) should work fine for your purposes. You can then execute searches with:

Tire.search(INDEX_NAME) { query { string 'hello world' } }
@Will-Sommers

This comment has been minimized.

Show comment Hide comment
@Will-Sommers

Will-Sommers Feb 18, 2013

Just to note, I've had to write a shell script to create my index with the proper analyzers and scoring. When importing multiple types into a single index only the first model takes settings in my experience.

Just to note, I've had to write a shell script to create my index with the proper analyzers and scoring. When importing multiple types into a single index only the first model takes settings in my experience.

@ichilton

This comment has been minimized.

Show comment Hide comment
@ichilton

ichilton Apr 24, 2013

Can anyone point me in the direction of an explanation / more information regarding this please? -

# Optional, but recommended: use a single index per application per environment.
# Caveat: This convention not be entirely supported throughout Tire's API.

Is the default to have an index for each model?

What's the advantages/disadvantages of a single index for the app?

How many indexes do Bonsai allow?

Thanks,

Ian

Can anyone point me in the direction of an explanation / more information regarding this please? -

# Optional, but recommended: use a single index per application per environment.
# Caveat: This convention not be entirely supported throughout Tire's API.

Is the default to have an index for each model?

What's the advantages/disadvantages of a single index for the app?

How many indexes do Bonsai allow?

Thanks,

Ian

@aarongray

This comment has been minimized.

Show comment Hide comment
@aarongray

aarongray May 21, 2013

This was a helpful gist, thanks for posting it up.

This was a helpful gist, thanks for posting it up.

@nz

This comment has been minimized.

Show comment Hide comment
@nz

nz May 21, 2013

@ichilton

Is the default to have an index for each model?

This is Tire's default, yes.

What's the advantages/disadvantages of a single index for the app?

Each index carries its own overhead, and multiple indices can be wasteful for small applications. It is negligible, but in a shared cluster environment, that adds up. Creating an index per model can help as an additional level of natural partitioning, but in most cases that is premature optimization, and you will be better served by consolidating into a single index per environment.

The main disadvantage for using a single index with Tire is that it doesn't always fit within Tire's conventions. If you ever have trouble related to index naming conventions, you should just stick with Tire's defaults.

How many indexes do Bonsai allow?

The total primary shards varies per plan. All plans should be able to comfortable index 2–3 models. We're still working on exactly where to set those numbers, so email me at support@bonsai.io if you ever bump into those limits.

@aarongray

Thanks! Happy to help.

Owner

nz commented May 21, 2013

@ichilton

Is the default to have an index for each model?

This is Tire's default, yes.

What's the advantages/disadvantages of a single index for the app?

Each index carries its own overhead, and multiple indices can be wasteful for small applications. It is negligible, but in a shared cluster environment, that adds up. Creating an index per model can help as an additional level of natural partitioning, but in most cases that is premature optimization, and you will be better served by consolidating into a single index per environment.

The main disadvantage for using a single index with Tire is that it doesn't always fit within Tire's conventions. If you ever have trouble related to index naming conventions, you should just stick with Tire's defaults.

How many indexes do Bonsai allow?

The total primary shards varies per plan. All plans should be able to comfortable index 2–3 models. We're still working on exactly where to set those numbers, so email me at support@bonsai.io if you ever bump into those limits.

@aarongray

Thanks! Happy to help.

@ckreutz

This comment has been minimized.

Show comment Hide comment
@ckreutz

ckreutz Jul 19, 2013

I have the same problem as @pawel2105 had and wonder what is the solution? Via curl locally all works fine. But I get this error when I try to import.

→ heroku run rake environment tire:import CLASS=Initiative FORCE=true
Running rake environment tire:import CLASS=Initiative FORCE=true attached to terminal... up, run.5891
[IMPORT] Deleting index 'stadt-gestalten-production'
rake aborted!
Server broke connection
/app/vendor/bundle/ruby/2.0.0/gems/rest-client-1.6.7/lib/restclient/request.rb:182:in rescue in transmit' /app/vendor/bundle/ruby/2.0.0/gems/rest-client-1.6.7/lib/restclient/request.rb:140:intransmit'
/app/vendor/bundle/ruby/2.0.0/gems/rest-client-1.6.7/lib/restclient/request.rb:64:in execute' /app/vendor/bundle/ruby/2.0.0/gems/rest-client-1.6.7/lib/restclient/request.rb:33:inexecute'
/app/vendor/bundle/ruby/2.0.0/gems/rest-client-1.6.7/lib/restclient.rb:88:in head' /app/vendor/bundle/ruby/2.0.0/gems/tire-0.6.0/lib/tire/http/client.rb:43:inhead'
/app/vendor/bundle/ruby/2.0.0/gems/tire-0.6.0/lib/tire/index.rb:18:in exists?' /app/vendor/bundle/ruby/2.0.0/gems/tire-0.6.0/lib/tire/tasks.rb:14:increate_index'
/app/vendor/bundle/ruby/2.0.0/gems/tire-0.6.0/lib/tire/tasks.rb:110:in `block (3 levels) in <top (required)>'
Tasks: TOP => tire:import => tire:import:model

ckreutz commented Jul 19, 2013

I have the same problem as @pawel2105 had and wonder what is the solution? Via curl locally all works fine. But I get this error when I try to import.

→ heroku run rake environment tire:import CLASS=Initiative FORCE=true
Running rake environment tire:import CLASS=Initiative FORCE=true attached to terminal... up, run.5891
[IMPORT] Deleting index 'stadt-gestalten-production'
rake aborted!
Server broke connection
/app/vendor/bundle/ruby/2.0.0/gems/rest-client-1.6.7/lib/restclient/request.rb:182:in rescue in transmit' /app/vendor/bundle/ruby/2.0.0/gems/rest-client-1.6.7/lib/restclient/request.rb:140:intransmit'
/app/vendor/bundle/ruby/2.0.0/gems/rest-client-1.6.7/lib/restclient/request.rb:64:in execute' /app/vendor/bundle/ruby/2.0.0/gems/rest-client-1.6.7/lib/restclient/request.rb:33:inexecute'
/app/vendor/bundle/ruby/2.0.0/gems/rest-client-1.6.7/lib/restclient.rb:88:in head' /app/vendor/bundle/ruby/2.0.0/gems/tire-0.6.0/lib/tire/http/client.rb:43:inhead'
/app/vendor/bundle/ruby/2.0.0/gems/tire-0.6.0/lib/tire/index.rb:18:in exists?' /app/vendor/bundle/ruby/2.0.0/gems/tire-0.6.0/lib/tire/tasks.rb:14:increate_index'
/app/vendor/bundle/ruby/2.0.0/gems/tire-0.6.0/lib/tire/tasks.rb:110:in `block (3 levels) in <top (required)>'
Tasks: TOP => tire:import => tire:import:model

@nz

This comment has been minimized.

Show comment Hide comment
@nz

nz Jul 19, 2013

Hi @criscrossed, you should email support@bonsai.io with your account information so I can check that out.

Owner

nz commented Jul 19, 2013

Hi @criscrossed, you should email support@bonsai.io with your account information so I can check that out.

@nz

This comment has been minimized.

Show comment Hide comment
@nz

nz Oct 20, 2013

Updated the setup instructions to remove the "optional" combined index strategy. Our plans offer sufficient shards these days to support most common index-per-model sharding strategies.

Owner

nz commented Oct 20, 2013

Updated the setup instructions to remove the "optional" combined index strategy. Our plans offer sufficient shards these days to support most common index-per-model sharding strategies.

@ralphamale

This comment has been minimized.

Show comment Hide comment
@ralphamale

ralphamale Jun 26, 2014

@nz - have you switched over to elasticsearch-rails gem? i'm trying to install it and it's giving me problems with Faraday.

@nz - have you switched over to elasticsearch-rails gem? i'm trying to install it and it's giving me problems with Faraday.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment