Skip to content

Instantly share code, notes, and snippets.

@rugyoga
Last active July 21, 2022 19:22
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save rugyoga/67a457a59a4925eab343862d2cca9c1e to your computer and use it in GitHub Desktop.
Save rugyoga/67a457a59a4925eab343862d2cca9c1e to your computer and use it in GitHub Desktop.
Create PIT ID, perform multiple ES searches and then return a Stream of hits.
@spec stream_many(ES.query_t(), ES.index_t(), ES.consumer_t(), non_neg_integer()) :: {:error, any()} | {:ok, Enumerable.t()}
def stream_many(query, index, consumer, count) do
case PIT.create(index) do
{:ok, pit_id} ->
try do
case pit_id |> Query.initial(query) |> streamer() do
{:ok, stream} ->
stream
|> Stream.flat_map(& &1.body["hits"]["hits"])
|> Stream.take(count)
|> then(consumer)
error -> error
end
after
PIT.delete(pit_id)
end
error -> error
end
rescue
error -> {:error, error}
end
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment