Skip to content

Instantly share code, notes, and snippets.

@marianogappa
marianogappa / ordered_parallel.go
Last active Aug 9, 2021
Parallel processing with ordered output in Go
View ordered_parallel.go
/*
Parallel processing with ordered output in Go
(you can use this pattern by importing https://github.com/MarianoGappa/parseq)
This example implementation is useful when the following 3 conditions are true:
1) the rate of input is higher than the rate of output on the system (i.e. it queues up)
2) the processing of input can be parallelised, and overall throughput increases by doing so
3) the order of output of the system needs to respect order of input
- if 1 is false, KISS!
@marianogappa
marianogappa / backpressure.go
Created Dec 4, 2016
Example backpressure implementation in Go
View backpressure.go
/*
This snippet is an example of backpressure implementation in Go.
It doesn't run in Go Playground, because it starts an HTTP Server.
The example starts an HTTP server and sends multiple requests to it. The server starts denying
requests by replying an "X" (i.e. a 502) when its buffered channel reaches capacity.
This is not the same as rate-limiting; you might be interested in https://github.com/juju/ratelimit
or https://godoc.org/golang.org/x/time/rate.