RapGenius has an interesting post about Heroku's randomized load balancing, complaining about how random placement degrades performance compared to prior omniscient approaches. RapGenius ran some simulations, including an experiments with a "Choice of Two" method:
Choice of two routing is the naive method from before, with the twist that when you assign a request to a random dyno, if that dyno is already busy then you reassign the request to a second random dyno, with no regard for whether the second dyno is busy
This differs subtly but substantially from the standard "Power of Two Choices" randomized load balancing:
each [request] is placed in the least loaded of d >= 2 [Dynos] chosen independently and uniformly at random
Take a look at the difference in queue lengths below, for 200 Dynos, 100