go circuit breakers

circuit_breakers.md

github.com/eapache/go-resiliency/breaker

func New(errorThreshold, successThreshold int, timeout time.Duration) *Breaker

New constructs a new circuit-breaker that starts closed. From closed, the breaker opens if "errorThreshold" errors are seen without an error-free period of at least "timeout". From open, the breaker half-closes after "timeout". From half-open, the breaker closes after "successThreshold" consecutive successes, or opens on a single error.

Usage:

for {
    result := breaker.Run(func() error {
        // communicate with some external service and
        // return an error if the communication failed
        return nil
    })

    switch result {
    case nil:
        // success!
    case ErrBreakerOpen:
        // our function wasn't run because the breaker was open
    default:
        // some other error
    }
}

https://github.com/sony/gobreaker

func NewCircuitBreaker(st Settings) *CircuitBreaker

type Settings struct {
    Name          string
    MaxRequests   uint32
    Interval      time.Duration
    Timeout       time.Duration
    ReadyToTrip   func(counts Counts) bool
    OnStateChange func(name string, from State, to State)
}

Name is the name of the CircuitBreaker.

MaxRequests is the maximum number of requests allowed to pass through when the CircuitBreaker is half-open. If MaxRequests is 0, the CircuitBreaker allows only 1 request.

Interval is the cyclic period of the closed state for the CircuitBreaker to clear the internal Counts. If Interval is 0, the CircuitBreaker doesn't clear internal Counts during the closed state.

Timeout is the period of the open state, after which the state of the CircuitBreaker becomes half-open. If Timeout is 0, the timeout value of the CircuitBreaker is set to 60 seconds.

ReadyToTrip is called with a copy of Counts whenever a request fails in the closed state. If ReadyToTrip returns true, the CircuitBreaker will be placed into the open state. If ReadyToTrip is nil, default ReadyToTrip is used. Default ReadyToTrip returns true when the number of consecutive failures is more than 5.

OnStateChange is called whenever the state of the CircuitBreaker changes.*

Example: https://github.com/sony/gobreaker/blob/master/example/http_breaker.go

Other features:

• Exposes state via func (cb *CircuitBreaker) State() State

github.com/streadway/handy/breaker

func NewBreaker(failureRatio float64) Breaker

type Breaker interface {
    Allow() bool
    Success(time.Duration)
    Failure(time.Duration)
}

const (
    // DefaultWindow is the default number of per-second buckets that will be
    // considered when calculating metrics on the circuit breaker.
    DefaultWindow = 5 * time.Second

    // DefaultCooldown is the default period a circuit will remain in the open
    // state before allowing a single sentinel request through.
    DefaultCooldown = 1 * time.Second

    // DefaultMinObservations is the default number of observations that must
    // be made before the circuit breaker
    DefaultMinObservations = 10
)

NewBreaker constructs a new circuit breaker, initially closed. The breaker opens after failureRatio failures per success, and only after DefaultMinObservations have been made.

Other notes:

The breaker will not open if there are less then 10 (MinObservations) values to judge on (even if 100% failed) If 50% (failureRatio) of all requests in the last 5 seconds (DefaultWindow) fail, the breaker will open After 1 second (coolDown), the breaker will allow one request again, to check availability

Other features:

• Exposes manual recording of Success/Failure (other ones expose wrappers like Run(func(error)))
• Exposes functions that return http.Handler and http.RoundTripper for circuit breaking in middleware and clients.

github.com/afex/hystrix-go/hystrix

Example:

hystrix.ConfigureCommand("my_command", hystrix.CommandConfig{
	Timeout:                1000,
	MaxConcurrentRequests:  100,
	RequestVolumeThreshold: 20,
	SleepWindow:            5000,
	ErrorPercentThreshold:  25,
})

hystrix.Do("my_command", func() error {
	// talk to other services
	return nil
}, func(err error) error {
	// fallback for when services are down
	return nil
})

Timeout is how long to wait for command to complete, in milliseconds.

MaxConcurrentRequests is how many commands of the same type can run at the same time.

RequestVolumeThreshold is the minimum number of requests needed before a circuit can be tripped due to health.

SleepWindow is how long, in milliseconds, to wait after a circuit opens before testing for recovery.

ErrorPercentThreshold causes circuits to open once the rolling measure of errors exceeds this percent of requests.

Other features:

• Exposes SSE endpoint for dashboards to pull circuit status (e.g., hystrix-dashboard).
• Could use hystrixjs in JS clients to have same settings and expose the same SSE endpoint.
• Max concurrent requests is a nice sanity check

github.com/go-kit/kit/circuitbreaker

Wrapper around either sony/gobreaker, handy/breaker, and hystrix-go (user's choice).