Small note of gRPC Best Practice @ CoreOSFest 2017

NOTE.md

gRPC Best Practice @ CoreOSFest 2017

https://coreosfest2017.sched.com/event/AWYc/best-practices-for-go-grpc-services-doug-fawley-google

Just small note for me. Hope the slide will be opened.

• API design
  • Idmpotency
    • It should be safe to retry an RPC without knowing whether it was processed
    • Example
      • request: add timestamp or GUID
      • response: confirmation (always same)
  • Performance
    • Repeated filed
      • request: can imply unbounded work --set limits
      • response: API should typically support pagination
    • Avoid long running operations
      • The longer it takes, the more likely it will need to be retried
      • Perform in background - send results asynchronously
        • Callbacks, email, pubsub, etc) or provide a tracking token to allow user to check
  • Default
    • define sensible default behavior
      • unset enums default to the zero value
        • prefer UNKNOWN, UNSPECIFILED, UNSET as the default
    • backward compatiblitity
      • even if you control the clients, you can’t always synchronously update client and server
  • Error
    • Error
      • First class in gRPC
        • payload or error
      • Do not include in response payload in most cases
        • client logic will be complicated ...
    • Avoid batching multiple, independent operations
      • e.g. updating many records in a database
      • error handling will becomes complex...
      • Use a streaming RPC or multiple calls instead
        • grpc is cheap
• Error handling
  • don’t panic!
    • handle errors as gracefully as possible
  • proper propagation
    • don’t blindly return error from libraries or other services
• Dead lines
  • Deadlines allows both severs and clients to know when to abort an operation
  • Always use dadlines
  • clients are responsible for setting deadlines!
  • Services also care about deadlines (check request)
    • Too short: not enough time to perform
    • Too long: consume resource
  • Propagation
    • Your service is often also a client (or many)
    • Method1
      • reuse context
      • pros: simple
      • cons: may wait longer than necessary
    • Method2
      • use tight deadlines on all outgoing calls
      • pros: simple, always early cancellation
      • cons:pessimistic
    • Method3 <-ideal
      • Use optimistic assumptions about remaining work

 d, _ := ctx.Deadline()
 ctx1, cancel := context.WithDeadline(ctx, d.Add(-150*time.Millisecond))

• Rate limiting
  • Servers are responsible for enforcing local rate limits
    • grpc.IntapHandle(rateLimitter) based on user map
  • Well behaved clients should implement local rate limiting, matching server enforcement
• Retries
  • Official feature planned: gRFC A6
    • Configurd via Servie Config
    • Supports
      • sequential retires with backoff, or
      • concurrent hedged requests
  • Until then: use a client wrapper or interceptor to perform retries
    • Accept a context and use it as deadline
  • Implement generic wrapper lib or interceptor
• Memory management
  • gRPC-go doesnot limit concurrent server goroutines
    • Set Listener limits: use netutil.LimitListener
    • use Tap Handler to error when too many RPCs are in flight
      • or when memory is too low
    • use health reporting and locald balancers to redirect traffic
  • Set max request palyload size in server
  • small requests can require massibve response size
    • API design issue: use streaming response in thi case
• Logging
  • Debugging after a problem has been detected
• Monitoring
  • Export metrics for everything
  • Capture request info

Question about authentication/authorization after session

• Use https://github.com/grpc/grpc-go/blob/master/credentials/credentials.go