-
-
Save fcheung/92d77b62eb570031823d to your computer and use it in GitHub Desktop.
module Tire | |
module Search | |
class Search | |
attr_reader :types | |
end | |
class MultiSearch | |
def initialize(searches, options ={}) | |
@searches = searches | |
@options = options | |
end | |
def to_payload | |
payload = @searches.each_with_object([]) do |search, result| | |
result << {:index => search.indices, :types => search.types}.to_json | |
result << search.to_json | |
end | |
payload << "" | |
payload.join("\n") | |
end | |
def results | |
@results || (perform; @results) | |
end | |
def perform | |
response = Configuration.client.post("#{Configuration.url}/_msearch", to_payload) | |
json = MultiJson.decode(response.body) | |
@results = json['responses'].each_with_index.collect do |response_json, index| | |
Results::Collection.new(response_json, @options.merge(@searches[index].options)) | |
end | |
end | |
end | |
end | |
end |
I think that it would be nice to have both the declarative form (my version), as well as assembled ad-hoc form (as in this gist). I'd also add the << method to the msearch above.
@fcheung: By the way, is there any observable/measurable benefit from multi search, compared to HTTP client which uses persistent connection?
I don't know to be honest - haven't tried that yet
@fcheung: Would be awesome if you could do a simple benchmark à la https://gist.github.com/1204159 ...
Can do - didn't realise that tire already had support for multiple http clients built-in
I'll make a proper reproducible benchmark later on (I'll also look into using ipfw to simulate the difference as latency increases), but just plugging this into my app I get around 120ms using rest client, individual searches, around 80ms using curb, 30ms using rest client + msearch, and curb + msearch is around the same.
@fcheung it was just recently merged :) We're using it with net-http-persistent, and meanwhile quite happy with it. We verified that it indeed uses persistent connections.
it was just recently merged
@romanbsd: No, Tire had support for RestClient and Curb for a long long time (karmi/retire@8010531)
@fcheung: Those numbers sound very interesting.
@karmi oops, I was referring to persistent connections, I didn't succeed in achieving persistent conns before that.
@romanbsd: Curb definitely does persistence connections, see the benchmark...
I was checking it with tcpdump at the time, and was observing SYN packets. Perhaps there was something different in my setup... Is curb's persistent connection handling thread safe?
Will this get rolled into Tire? For now I am gonna use the bits I need to make a multisearch request via RestClient manually until that happens.
Yes, multi search support is now available in Tire 0.5.0/master:
Personally I'd return a hash or an array (depending on whether you gave an array or a hash of queries). I'm not sure that a wrapper would add a lot here - there doesn't seem to be any metadata from elastic search that refers to the batch as a whole