Skip to content

Instantly share code, notes, and snippets.

@pbyrne

pbyrne/day-1.markdown

Created July 21, 2014 15:24
Show Gist options
  • Save pbyrne/0bbb800586e1c9b32843 to your computer and use it in GitHub Desktop.
Save pbyrne/0bbb800586e1c9b32843 to your computer and use it in GitHub Desktop.
Download ZIP
My conference notes from DevOpsDays Minneapolis 2014.
Raw
day-1.markdown

Presentations

Opening Keynote

  • Sascha Bates
  • Themes of DevOps:
    • Empathy
    • Trust
    • Reflection
    • INCLUSIVENESS!

The Skeleton of DevOps

  • Dan Slimmon
  • Is DevOps working?
    • Depends on what you wanted to get
    • Then you can measure what happened vs what you wanted
  • Have a theory:
    • A set of rules
    • That generates predictions
    • About the behavior of a system
  • Then you can experiment
    • Experiments weed out bad theories
    • Theories tell us what experiments to try
  • Conway's Law
    • 1968
    • Organizations create systems which mirror the organization's communication structure
  • We continually redesign our software, systems, etc.
  • So too should we redesign our team structure
    • Some organizations have been successful taking the reverse of Conway's law: reorganize your communication structure to let the application structure you want fall out
  • Destroy the wall of confusion between Dev & Ops
  • Theory is a conversation
    • Never done, always evolving
    • Geocentricism -> Heliocentricism -> Theory of Gravity -> Landing on the Moon

When Dev Met Ops

  • Heather and Ross from Target
    • Explaining from a Dev and an Ops perspective how DevOps benefited Target
  • Lot of technology, 50 years of complexity accrued
  • Lots of silos
    • Silos within silos
  • Remove roadblocks to move forward
  • Create a safety zone
    • Make it safe to fail

Handmade Security at Etsy

  • Ben Hughs
    • Etsy security team
    • @benjammingh
  • People don't come into your servers through the front door
    • Easy to protect
    • Look in the non-obvious places, like your developers' laptops becoming compromised
  • You could cause everything to stop when something suspicious happens
    • But too many false alarms
  • Better to log everything
    • Then you can review after-the-fact and close down whatever happened
  • Blameless post-mortem
  • Seriously, log everything

Open Sessions

Log Aggregation (OSS)

  • Splunk is popular and full featured
    • It's super expensive
  • Logstash is an open-source project that covers much of the same ground
    • not designed as a Splunk clone/replacement
    • send log entries to it for later searching
    • also alerting on spikes of things and whatnot
    • not as full-featured as Splunk
    • Uses elastic search for searching
    • Easy to scale horizontally
      • master/search/data on one server
      • master on one server, multiple search servers, multiple clustered data servers
  • Mostly the conversation turned to people comparing and contrasting Splunk/Logstash
    • What they had to do in Logstash to replace Splunk functionality
    • What is easier vs what is harder between them
    • I kind of checked out
  • Someone mentioned Sumo Logic as a SaaS alternative to both

Docker

  • It's becoming clear to me based on this conversation that Docker is most interesting/useful to someone running their own physical hardware, as an easier alternative to running VMs on the hardware.

Maintaining Culture of the Dev Team as it Grows

  • Basically, keep all lines of communication open
  • Be proactive including people outside your team
  • If documentation is poor, make it part of acceptance critiera
    • Documentation doesn't have to be amazing, just has to exist (can always improve later)
    • As part of a bug fix, improve documentation that lead to misunderstand that created the bug
    • As part of your on-call alerting, create a "play book" for each alert that outlines common causes/solutions
      • Makes it much easier for new on-call staff to be productive in case of an emergency
Raw
day-2.markdown

presentations

ensuring digital quality with promise theory

  • i didn't really understand this one

devops is dead. long live devops

  • katherine daniels
    • @beerops
  • outlining her evolution from cranky sysasmin to devops evangelist to having a more-nuanced understanding and appreciation of devops
  • used the hype cycle to explain her journey
  • of all of the tennants of devops to honor: empathy

fallible humans

  • ian malpass
    • developer at etsy
    • @indec
  • "dealing with failure in absence of scapegoats"
  • scapegoats have existed since antiquity
    • ancient jerusalem had literal goats, one to sacrifice and one (called the scapegoat) which was released into the wilderness to die
    • ancient greeks killed beggars and criminals to the gods if things went poorly
  • natural instinct is to blame an "other"
    • dev, ops, support, legal, finance, etc. can all be others in their silos
    • it's their fault
    • other "others": gender, educational background
  • blame leads to fear
  • unhealthy
    • but worse, it's ineffective
  • instead, everyone's on the same team: dev, ops, support, finance, etc. are all one big happy silo
  • "any explanation is better than none" - nietzche
    • false
  • failure is an emergent behavior of a complex system
  • if failure is inevitable, punishing at best doesn't work, at worst make things worse
  • we should learn from our mistakes, not punish
  • humans are good at balancing conflicting goals
    • often, by trading between efficiency and thoroughness
  • blame is ludirous
    • instead ask "why?"
    • that culture leads to trust and understanding
  • blame hides opportunities to learn
  • don't blame yourself
    • instead ask "why did i think that was a good idea?"
  • "there is no root cause"
    • typically many small failures contribute
    • each necessary but not sufficient to cause failure on their own
  • seek good remediation items
    • actions going forward to prevent failure
    • be wary of extra process
    • ensure that the items aren't worse than the failure
      • totally acceptable that "no action" is appropriate, that the failure that occurred is easier to accept than the effort to prevent it

5 years of devops

  • closing keynote
  • patrick dubois
    • started first devopsdays in belgium
  • i honestly didn't follow much of this, either
    • kind of a retrospective of what lead to the devops movement and reflection on what has changed and what hasn't since then
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment