Skip to content

Instantly share code, notes, and snippets.

Last active May 23, 2017 15:15
Show Gist options
  • Save chris-short/8e50511c8ff808f032a968eac41f5363 to your computer and use it in GitHub Desktop.
Save chris-short/8e50511c8ff808f032a968eac41f5363 to your computer and use it in GitHub Desktop.
A Night of DevOps - Open Source South Carolina


  1. Chris Short
  2. Global DevOps Engineer, SolarWinds MSP
  3. Triangle DevOps Organizer
  4. and DZone Contributor
  5. DevOpsDays Speaker and Organizer
  6. Founder of DevOps'ish Newsletter

What is DevOps?

Not everyone agrees on this

  1. Take a Deep Breath
  2. We are all allowed to have an opinion about this

What is DevOps? (cont'd)

What DevOps Isn't

  1. Let's rule some things out first
  1. I think this tweet puts to bed some IT misnomers

What DevOps Isn't (cont'd)

  1. Not Cloud
  2. Not a Tool or a Product You Can Buy
  3. Not Job Title or Position
  4. Not Department

So What is DevOps?!?


DevOps is The Three Ways Applied to Three Things

  1. This is what DevOps is to me...
  2. "The Three Ways" applied to "The Three Things"
  3. Hippy enough for you?

The Three Ways

  1. Let's go over "The Three Ways" first

The Three Ways (cont'd)

  1. Origin: The Pheonix Project by Kevin Behr, George Spafford, and Gene Kim
  2. Backstory: The Pheonix Project authors state the “Three Ways,” are the principles that all of the DevOps patterns can be derived from.
  3. Let's discuss each way further

The First Way

  1. Entire System Flow
  2. Not One Box or Silo

The First Way (cont'd)

  1. This is a broken First Way
  2. This is how a lot of IT organization operate

The First Way (cont'd)

  1. Who has run in to this problem?
  2. Developers say it works, Ops says it's broken

The Second Way

  1. Enabling Feedback
  2. Amplifying Feedback

The Second Way (cont'd)

  1. John Willis sent me this to represent all Three Ways
  2. I removed a couple components to have it apply better to The Second Way
  3. Ultimately business is what matters
  4. Getting more information to developers faster and into products faster makes business more competitive

The Third Way

  1. Continual Learning and Experimentation
  2. This should be the goal of every IT organization
  3. This is the process in which developers and operations are iterating small improvements to system and they are being pushed out to production through an often automate process

The Third Way (cont'd)

  1. This is The Third Way infinity loop
  2. A process of Continuous Integration and Continuous Deployment (CI/CD)
  3. This is very achievable and it's a great place to be

The Three Things

  1. People
  2. Processes
  3. Tools

Applying "The Three Ways" to "The Three Things"

  1. Let's dive into how "The Three Ways" applies to "The Three Things"


  1. Let's tackle the hardest thing first and talk about the people part of DevOps

Joining Flows

  1. Develop
  2. Operations
  3. To achieve The Third Way you should build cross-functional teams of systems, network, QA, devs, etc.

Same Team, Same Fight

  1. Developers
  2. Operations
  3. QA
  4. PMs
  5. Working Together Toward Common Goals

Communication/Collaboration (Expediting Person-to-Person Feedback)

  1. Devs and Ops working together regularly
  2. Example: Collaborative Scrum Teams
  3. Slack, FlowDock, IRC, etc.
  4. Managers aren't the only conduit between teams
  5. Everyone is a conduit

Passion, Pride, Ownership

  1. Team members must inspire passion in each other's work
  2. Individuals must take pride in their work and what the team is achieving
  3. Teams must take ownership in what they produce
  4. Highlight accomplishments! We overcame X to get to where we are today!
  5. Quantify in business terms like dollars and hours saved

Diverse backgrounds bring diverse solutions

  1. Different upbringings breed different people
  2. Different paths to get where your team is results in different ideas of how to achieve goals

A Case Study (Kinda)

A Case Study (Kinda) (cont'd)


A Case Study (Kinda) (cont'd)

  1. Five very different characters:
    1. Rey
    2. Han Solo
    3. Chewbacca
    4. Finn
    5. BB-8
  2. From all walks of life (or non-life in BB-8's case)
  3. Fixing Problems as They Arise
    1. Triaging Chewbacca
    2. Fixing the Millennium Falcon
  4. Building Trust in Each Other
  5. Going on to save the galaxy

The Psychology of it All

  1. Meat is Hard
  2. "I'm sorry. Would you two mind changing seats?"

People Resist Change

  1. Assumptions are hard to break
  2. Studies show that even if a new way is better people will not change unless properly incentivized

Game Theory

  2. I'm not a Game Theory expert (AT ALL)
  3. I got this from Nirmal Mehta who got this from Andrew Clay Shafer; Do Not Reproduce Without Permission

Who Are Pareto and Nash?

  1. Pareto: Italian engineer, sociologist, economist, political scientist, and philosopher
  2. Known for the 80/20 rule, named after him as the Pareto principle
  3. Nash: American mathematician who made fundamental contributions to game theory, differential geometry, and the study of partial differential equations.
  4. Nash's work has provided insight into the factors that govern chance and decision-making inside complex systems found in everyday life.

Pareto Efficiency

  1. An allocation of resources in which it is impossible to make any one individual better off
without making at least one individual worse off (KIND OF LIKE ZERO SUM)

Pareto Inefficiency

  1. A situation is inefficient if someone can be made better off even after compensating those made worse off.

Nash Equilibrium

  1. A concept of game theory where the optimal outcome of a game is one where no player has an incentive to deviate from his chosen strategy after considering an opponent's choice.

Pareto Inefficient Nash Equilibrium

  1. A Nash equilibrium is what's strategically feasible.
  2. A Pareto optimal solution is what's efficient.

Pareto Inefficient Nash Equilibrium (cont'd)

  1. Possible to make an individual better off
  2. Without making any individual worse off
  3. But no one will change their strategy...

Pareto Inefficient Nash Equilibrium (cont'd)

  1. ...Until the game changes


  1. People are difficult
  2. How can we help them?

How Do We Change the Game in our Organizations?

Remember We Are Trying to Get Here!

  1. Tight Feedback Loops
  2. Failing Fast
  3. What does Failing Fast mean to you?
  4. What Processes Can We Put In Place to Enable This?
  5. How are we going to get people to embrace the new process?

Incentivize the new way of work

  1. People resist change, right?
  2. If you have to use the stick and the carrot you aren't doing it right

I prefer the ALL CARROT approach

  1. Make people's lives easier
  2. Make people enjoy their work more
  3. Example: No CCB if you release through new pipeline

Implement Automation

  1. Build an attack plan
  2. Break the work into numerous, small, manageable pieces
  3. Release Frequently
  4. Remember: You are building a pipeline to speed business approval
  5. Measure your results; metrics are key
  6. Remember there will always be more work to automate


  1. Openness and honesty is KEY
  2. Engineering Blogs (Internal or External)
  3. Publicly Readable Code Repos
  4. ChatOps

Infrastructure as Code

  2. You can build pipelines in your own datacenter too
  3. Think of each step in this process as a feedback loop
  4. We want to automate as much of this as possible
  5. Ops can enable Devs and vice versa if we work together

Infrastructure as Code (cont'd)

  1. In the Infrastructure as Code model we must think bigger than individual servers/VMs
  2. DNS not /etc/hosts
  3. Service Discovery not Static Configuration
  4. Cattle not Pets

Think Globally

  1. Would you micro-manage your team members?
  2. Why are you micro-managing your servers then?
  3. Create an environment where services can flourish through your pipelines

This is the goal

  1. Enable push button releases
  2. Developers pushing code and off it goes into production


Wait! You said DevOps isn't a tool!

Wait! You said DevOps isn't a tool (cont'd)

  1. DevOps is not anyone one tool
  2. DevOps has common tooling but often the tools represent a formalized way to do something
  3. The Three Ways have to be applied to the tooling you use too



Version Control

  1. Many benefits:
    1. Traceability
    2. Backup
    3. Collaboration
    4. Feedback (hooks)
  2. git with all its warts has won the version control battle
  3. GitHub, GitLab, Apache Allura
  4. Everything must be in version control
  5. master/trunk should always be production ready

"Your system can't tell you what is not in Version Control" –Nicole Forsgren

  1. This is the biggest reason version control is important
  2. "Your system can't tell you what is not in Version Control" –Nicole Forsgren
  3. Random configuration or dependency not in version control is a recipe for disaster

Configuration Management

  1. Templating your configs is VERY handy
  2. Variables are awesome


  1. Orchestration tools are literally the conductor
  2. Glues all the pieces together
  3. Jenkins is the Swiss Army knife here


  1. This one slide could be a talk all by itself
  2. White-box monitoring (Internal)
  3. Black-box monitoring (External)
  4. Monitoring and Alerting should be separated
  5. Alerting tools need to be flexible to your people and your systems


  1. Your tools should talk
  2. If automation is running everyone should know
  3. If you are pushing changes the business should know
  4. If you have changed something there should be a ticket
  5. git should talk to JIRA
  6. Your orchestrator should talk to chat
  7. Your monitoring should talk to alerting AND chat AND a status page
  8. ChatOps
  9. The third rail of ChatOps is doing most of your production work in chat!

DevOps Booksmarks

  1. If you are looking for a tool to do something there are two useful resources
  2. DevOps Bookmarks is one


  1. Cloud Native Landscape is the holy grail (IMO)

When You Apply The Three Ways to The Three Things


  1. What does that mean?
  2. DevOps is a Culture!

Examples of DevOps

  1. Let's take a break
  2. When we come back we can talk about some practical examples of DevOps

"The Dark Side of DevOps"

  1. The Ops Mutiny
  2. The Dev Rebellion
  3. DevOps Strikes Back


  1. I can do this because I have Cloudflare in front of my website
  2. AWS
  3. Ansible
  4. Papertrail

LIVE DEMO!!! (THIS IS FINE) (cont'd)

DevOps is a Never-Ending Journey

Continuous Learning and Self-Improvement

  1. Continuous Learning and Self-Improvement is KEY!!!
  2. Meetups
  3. DevOpsDays Raleigh
  4. Newsletters
  5. Podcasts
  7. Hacker News

Continuous Learning and Self-Improvement (cont'd)

  1. The Phoenix Project is a novel
  2. DevOps Handbook is a guide
  3. I happen to have three copies of The Phoenix Project for y'all

Continuous Learning and Self-Improvement (cont'd)

  1. Shameless plug
  2. I contributed to an upcoming Open Organization Book
  3. Several great people in DevOps and Culture contributed pieces as well
  4. Coming June 2, 2017


Conclusion (cont'd)


Cutting Room Floor

Blameless Postmortems

  1. Team Designed and Implemented a Process/Product That Failed
  2. Not Any One Person's Fault
  3. Learn From Your Mistakes
  4. Share Documentation and Lessons Learned
  5. Improve
  6. Move-on

Influencers in DevOps Space

  1. Gene Kim
  2. Jez Humble
  3. Nicole Forsgren
  4. Bridget Kromhout
  5. Alice Goldfuss
  6. Josh Atwell
  7. Ashley McNamara
  8. Jessie Frazelle
  9. Charity Majors
  10. Julia Evans


Copy link

chris-short commented May 6, 2017

People - Passion, Ownership, Pride, Empathy

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment