Skip to content

Instantly share code, notes, and snippets.

@henrik
Forked from sasa1977/poolboy_demo.ex
Last active December 16, 2020 13:56
  • Star 25 You must be signed in to star a gist
  • Fork 11 You must be signed in to fork a gist
Star You must be signed in to star a gist
Save henrik/ceede9c4d9bf3fcb4dd5 to your computer and use it in GitHub Desktop.
Example of using Poolboy in Elixir to limit concurrency (e.g. of HTTP requests).
defmodule HttpRequester do
use GenServer
def start_link(_) do
GenServer.start_link(__MODULE__, nil, [])
end
def fetch(server, url) do
# Don't use cast: http://blog.elixirsips.com/2014/07/16/errata-dont-use-cast-in-a-poolboy-transaction/
timeout_ms = 10_000
GenServer.call(server, {:fetch, url}, timeout_ms)
end
def handle_call({:fetch, url}, _from, state) do
IO.puts "fetching #{url}"
:timer.sleep 5000
IO.puts "fetched #{url}"
{:reply, "whatever", state}
end
end
defmodule PoolboyDemo do
def run do
transaction_timeout_ms = 10_000
await_timeout_ms = 100_000
{:ok, pool} = :poolboy.start_link(
[worker_module: HttpRequester, size: 5, max_overflow: 0]
)
tasks = Enum.map 1..20, fn(n) ->
Task.async fn ->
:poolboy.transaction pool, fn(http_requester_pid) ->
HttpRequester.fetch(http_requester_pid, "url #{n}")
end, transaction_timeout_ms
end
end
tasks |> Enum.each &Task.await(&1, await_timeout_ms)
end
end
@henrik
Copy link
Author

henrik commented Sep 18, 2015

Changed these things from @sasa1977's version:

Using an :infinity timeout Using a custom, longer timeout when calling, and for poolboy's transactions (checking out a worker, doing something and then checking it back in), and when awaiting tasks. The default 5000 ms would easily be met otherwise.

@sasa1977
Copy link

Yes, this looks good and the changes make sense.
I suggest using GenServer.* instead of :gen_server.*

:infinity timeout is almost never a good idea, so I suggest using some finite value. Otherwise, you risk deadlocks and total blockages of your system.

@henrik
Copy link
Author

henrik commented Sep 18, 2015

@sasa1977 Good points both! Thank you for the review :D

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment