-
-
Save fcheung/92d77b62eb570031823d to your computer and use it in GitHub Desktop.
| module Tire | |
| module Search | |
| class Search | |
| attr_reader :types | |
| end | |
| class MultiSearch | |
| def initialize(searches, options ={}) | |
| @searches = searches | |
| @options = options | |
| end | |
| def to_payload | |
| payload = @searches.each_with_object([]) do |search, result| | |
| result << {:index => search.indices, :types => search.types}.to_json | |
| result << search.to_json | |
| end | |
| payload << "" | |
| payload.join("\n") | |
| end | |
| def results | |
| @results || (perform; @results) | |
| end | |
| def perform | |
| response = Configuration.client.post("#{Configuration.url}/_msearch", to_payload) | |
| json = MultiJson.decode(response.body) | |
| @results = json['responses'].each_with_index.collect do |response_json, index| | |
| Results::Collection.new(response_json, @options.merge(@searches[index].options)) | |
| end | |
| end | |
| end | |
| end | |
| end |
I think that would be pretty simple - just build up a hash of indexes to keys in the array that you then use to build up the result. If you assume order preserving hashes (ie ruby 1.9 only or pull in active support's ordered hash) ( what is tire's stance on that ? I seem to recall active support being a dependency) then it would be even simpler.
I don't think order matters here much? I think it ideally should be wrapped in some kind of enumerable, so you can access individual results by position or key, and iterate over them.
activesupport is not a direct dependency -- but activemodel pulls it in. I try to stay as far as possible from activesupport extensions.
(In the future, the Tire::Model::Persistence layer should be extracted to a separate extension or even a separate gem.)
The ordering matters only to be able to match up the response elements with the query that generated them.
Personally I'd return a hash or an array (depending on whether you gave an array or a hash of queries). I'm not sure that a wrapper would add a lot here - there doesn't seem to be any metadata from elastic search that refers to the batch as a whole
I think that it would be nice to have both the declarative form (my version), as well as assembled ad-hoc form (as in this gist). I'd also add the << method to the msearch above.
@fcheung: By the way, is there any observable/measurable benefit from multi search, compared to HTTP client which uses persistent connection?
I don't know to be honest - haven't tried that yet
@fcheung: Would be awesome if you could do a simple benchmark à la https://gist.github.com/1204159 ...
Can do - didn't realise that tire already had support for multiple http clients built-in
I'll make a proper reproducible benchmark later on (I'll also look into using ipfw to simulate the difference as latency increases), but just plugging this into my app I get around 120ms using rest client, individual searches, around 80ms using curb, 30ms using rest client + msearch, and curb + msearch is around the same.
@fcheung it was just recently merged :) We're using it with net-http-persistent, and meanwhile quite happy with it. We verified that it indeed uses persistent connections.
it was just recently merged
@romanbsd: No, Tire had support for RestClient and Curb for a long long time (karmi/retire@8010531)
@fcheung: Those numbers sound very interesting.
@karmi oops, I was referring to persistent connections, I didn't succeed in achieving persistent conns before that.
@romanbsd: Curb definitely does persistence connections, see the benchmark...
I was checking it with tcpdump at the time, and was observing SYN packets. Perhaps there was something different in my setup... Is curb's persistent connection handling thread safe?
Will this get rolled into Tire? For now I am gonna use the bits I need to make a multisearch request via RestClient manually until that happens.
Yes, multi search support is now available in Tire 0.5.0/master:
I like the approach where you declare the searches and then pass them to
Tire::Search::MultiSearch.Would be cool to be able to "name" the searches, so instead of
results[0]you could sayresults[:red_garments].There's a pull request for multisearch at https://github.com/karmi/tire/pull/308/files, but this feels much better.