Skip to content

Instantly share code, notes, and snippets.

@Auz
Last active November 23, 2020 02:16
Show Gist options
  • Save Auz/73a2158f4220799d32f3eda70479b398 to your computer and use it in GitHub Desktop.
Save Auz/73a2158f4220799d32f3eda70479b398 to your computer and use it in GitHub Desktop.
Build Vs Buy Table
Build Buy
Pro
  • Tight integration into your code base
  • Uses existing metrics and data, including custom metrics
  • Aware of advanced caching
  • Easy to target audiences
  • Almost immediately able to test
  • Proven statistics engine
  • Self-service front end testing admin - testing may done by non technical teams
  • Slick admin interface
  • Very easy to use
  • New features rolled out independently of your team
Con
  • High opportunity costs to build
  • High maintenance costs
  • Generally poor user experience
  • Takes months to build right
  • High risks for incorrect results
  • Have to build trust
  • No self-service front end testing
  • Engineering often required to implement tests
  • Ongoing subscription costs - some platforms can be expensive
  • Limited integration with code base
  • Yet another place for user data
  • 3rd party tracking often blocked
  • Most only support binomial metrics
  • Front end testing can cause flickering
  • Limited customization - not all tests can be run
  • Sometimes there can be “too many cooks” running A/B tests
|
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment