Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Example of using Poolboy in Elixir to limit concurrency (e.g. of HTTP requests).
defmodule HttpRequester do
use GenServer
def start_link(_) do
GenServer.start_link(__MODULE__, nil, [])
end
def fetch(server, url) do
# Don't use cast: http://blog.elixirsips.com/2014/07/16/errata-dont-use-cast-in-a-poolboy-transaction/
timeout_ms = 10_000
GenServer.call(server, {:fetch, url}, timeout_ms)
end
def handle_call({:fetch, url}, _from, state) do
IO.puts "fetching #{url}"
:timer.sleep 5000
IO.puts "fetched #{url}"
{:reply, "whatever", state}
end
end
defmodule PoolboyDemo do
def run do
transaction_timeout_ms = 10_000
await_timeout_ms = 100_000
{:ok, pool} = :poolboy.start_link(
[worker_module: HttpRequester, size: 5, max_overflow: 0]
)
tasks = Enum.map 1..20, fn(n) ->
Task.async fn ->
:poolboy.transaction pool, fn(http_requester_pid) ->
HttpRequester.fetch(http_requester_pid, "url #{n}")
end, transaction_timeout_ms
end
end
tasks |> Enum.each &Task.await(&1, await_timeout_ms)
end
end
@henrik

This comment has been minimized.

Copy link
Owner Author

@henrik henrik commented Sep 18, 2015

Changed these things from @sasa1977's version:

Using an :infinity timeout Using a custom, longer timeout when calling, and for poolboy's transactions (checking out a worker, doing something and then checking it back in), and when awaiting tasks. The default 5000 ms would easily be met otherwise.

@sasa1977

This comment has been minimized.

Copy link

@sasa1977 sasa1977 commented Sep 18, 2015

Yes, this looks good and the changes make sense.
I suggest using GenServer.* instead of :gen_server.*

:infinity timeout is almost never a good idea, so I suggest using some finite value. Otherwise, you risk deadlocks and total blockages of your system.

@henrik

This comment has been minimized.

Copy link
Owner Author

@henrik henrik commented Sep 18, 2015

@sasa1977 Good points both! Thank you for the review :D

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment