Skip to content

Instantly share code, notes, and snippets.

@wuxinwei
Forked from tcnksm/NOTE.md
Created October 13, 2017 11:40
Show Gist options
  • Save wuxinwei/5edd2a586b712a3430d48bc494a770d0 to your computer and use it in GitHub Desktop.
Save wuxinwei/5edd2a586b712a3430d48bc494a770d0 to your computer and use it in GitHub Desktop.
Small note of gRPC Best Practice @ CoreOSFest 2017

gRPC Best Practice @ CoreOSFest 2017

https://coreosfest2017.sched.com/event/AWYc/best-practices-for-go-grpc-services-doug-fawley-google

Just small note for me. Hope the slide will be opened.

  • API design
    • Idmpotency
      • It should be safe to retry an RPC without knowing whether it was processed
      • Example
        • request: add timestamp or GUID
        • response: confirmation (always same)
    • Performance
      • Repeated filed
        • request: can imply unbounded work --set limits
        • response: API should typically support pagination
      • Avoid long running operations
        • The longer it takes, the more likely it will need to be retried
        • Perform in background - send results asynchronously
          • Callbacks, email, pubsub, etc) or provide a tracking token to allow user to check
    • Default
      • define sensible default behavior
        • unset enums default to the zero value
          • prefer UNKNOWN, UNSPECIFILED, UNSET as the default
      • backward compatiblitity
        • even if you control the clients, you can’t always synchronously update client and server
    • Error
      • Error
        • First class in gRPC
          • payload or error
        • Do not include in response payload in most cases
          • client logic will be complicated ...
      • Avoid batching multiple, independent operations
        • e.g. updating many records in a database
        • error handling will becomes complex...
        • Use a streaming RPC or multiple calls instead
          • grpc is cheap
  • Error handling
    • don’t panic!
      • handle errors as gracefully as possible
    • proper propagation
      • don’t blindly return error from libraries or other services
  • Dead lines
    • Deadlines allows both severs and clients to know when to abort an operation
    • Always use dadlines
    • clients are responsible for setting deadlines!
    • Services also care about deadlines (check request)
      • Too short: not enough time to perform
      • Too long: consume resource
    • Propagation
      • Your service is often also a client (or many)
      • Method1
        • reuse context
        • pros: simple
        • cons: may wait longer than necessary
      • Method2
        • use tight deadlines on all outgoing calls
        • pros: simple, always early cancellation
        • cons:pessimistic
      • Method3 <-ideal
        • Use optimistic assumptions about remaining work
 d, _ := ctx.Deadline()
 ctx1, cancel := context.WithDeadline(ctx, d.Add(-150*time.Millisecond))
  • Rate limiting
    • Servers are responsible for enforcing local rate limits
      • grpc.IntapHandle(rateLimitter) based on user map
    • Well behaved clients should implement local rate limiting, matching server enforcement
  • Retries
    • Official feature planned: gRFC A6
      • Configurd via Servie Config
      • Supports
        • sequential retires with backoff, or
        • concurrent hedged requests
    • Until then: use a client wrapper or interceptor to perform retries
      • Accept a context and use it as deadline
    • Implement generic wrapper lib or interceptor
  • Memory management
    • gRPC-go doesnot limit concurrent server goroutines
      • Set Listener limits: use netutil.LimitListener
      • use Tap Handler to error when too many RPCs are in flight
        • or when memory is too low
      • use health reporting and locald balancers to redirect traffic
    • Set max request palyload size in server
    • small requests can require massibve response size
      • API design issue: use streaming response in thi case
  • Logging
    • Debugging after a problem has been detected
  • Monitoring
    • Export metrics for everything
    • Capture request info

Question about authentication/authorization after session

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment