Skip to content

Instantly share code, notes, and snippets.

@zachm
Last active October 13, 2016 18:44
Show Gist options
  • Save zachm/a6510d51f1b6288dcb6181603a33d0f5 to your computer and use it in GitHub Desktop.
Save zachm/a6510d51f1b6288dcb6181603a33d0f5 to your computer and use it in GitHub Desktop.
DevOpsDays DTW

Enter The Trough Of Disillusionment

Jim Drewes @ Daugherty Business Solutions

It was okay - I didn't take many notes though.

Gartner's hype curve strongly implies that, with all the shiny new devops tools, the "Trough of Disillusionment" is soon to follow.

Basically, Jim believes that due to quickly-approaching enterprise adoption, devops is about to be come "a hell of a lot less fun". This is probably the case, but I don't know that I agree completely - there's always going to be younger technologies and companies on the vanguard of the technical bits of the movement.

How You Can Help in Your Corner with Data Science: Drawing from Successes at the Detroit Fire Department

Cassie DeWitt @ City of Detroit

I wish I could say that I enjoyed this talk, but it was... just okay? Maybe I'm not the intended audience.

She's coming from an urban planning background. Detroit has issues with public services, so how can you use data science to improve service with tight resource constraints? For example, many FDs can't actually do all building inspections, so they use some metrics to prioritize where the important (scary) priorities should be.

"Sometimes there are problems, but they aren't mentioned in the meetings." They're discussed offline in traditional human relationships. So have the one on one discussions to help.

Oh, mentoring is important? Wow. I agree, it's definitely even more crucial in a legacy org like Detroit FD, that's kind of not tech first.

  • Operational problems --> reporting
  • Spatial problems --> mapping
  • Data quality problems --> scripting
  • Deployment problems --> intensive analysis
  • Problem solving --> intensive analysis"

^^ This seemed the most useful slide so I wrote it down.

Geospatial DB that logs fire risk and arson risk and unifies those to get a risk score. That's pretty cool.

  • Q: Metric you're most proud of?
  • A: She tried to convert some people from Excel to R, but that was hard. So instead she made Excel charts have prettier colors (has a design background). Changing an Excel template made them able to have better conversations about the data within it, and emphasized trends that otherwise wouldn't have been as visible.

Debugging TLS/SSL

John Downey @ Braintree/PayPal

These slides are very dense (perhaps unsurprisingly) so here is the deck: https://speakerdeck.com/jtdowney/ssl-at-devops-days-detroit-2016

"TLS 1.0 is only slightly different from SSL 3.0." Netscape worked on SSL, became a standard when IETF took it over.

Why do we need this?

  • Confidentiality
  • Integrity
  • Identification

Public key crypto: Gives you signatures and key exchange. Important for TLS, asymm crypto is way slower than symmetric crypto. Asymm starts the process and then we switch to symmetric.

PEM files: Privacy Enhanced Mail (but not used for that). The usual header you see all the time: "-----BEGIN CERTIFICATE-----" File extensions are lies, though: may describe format but not contents, or v/v. (.cer, .crt, .key, .pem).

OpenSSL: Rumor has it that Eric A Young taught himself C by writing OpenSSL...

X.509 Cert: Wraps public key, shows subject of cert and issuer, and digitally signed by issuer.

openssl x509 -text -in some_file.pem : learn all the things, his most useful command.

openssl s_client -connect www.braintreepayments.com:443 : will drop you to prompt where you can type out, by hand, your HTTP request. Pretty cool.

OpenSSL doesn't give you any root certs, but OS and browsers do. OS X root certs are ancient, so use OpenSSL from homebrew if you are on OS X.

sslyze: Great tool to analyze certificate trust, etc. Homebrew has it.

TLS 1.3 will have fewer round trips, which will very much improve performance.

SNI (extension post TLS 1.0) is very problematic. Client tells server what vhost they want; otherwise, every hostname needs its own IP. For a long while Python didn't support it.

The Most Dangerous Code in the World - Martin Georgiev's paper (he works at Yelp!) that talks about validating TLS in non-browser code.

Intermediate certs: browsers will trust them by following the chain, but other code won't because it's missing one of the certs and therefore can't validate.

Hostname Verification: Really, really horrible default settings!

Dev Owns Quality When Dev Feels The Pain

Jeremy Rickard @ VMWare

Homie's first talk! He did well. I think the topics he presented on are very useful, just we already do them!

Basically the idea is in the title: Developers care more about quality when they're on the hook to clean up their own messes. I think we're already doing pretty well on this, with product teams having on-call etc.

Better config management, repeatable environments, manageability. Yeah those are good things.

Auto-triage issues, especially the most common ones.

Ooh, add logging and monitoring! (Shocking...)

(At least on his team) CD with Jenkins, provisioning with Terraform. Integrations btwn Slack and Jenkins. Mesos and Docker, although he didn't talk about it in depth. Marathon load balancer. Blue/green deployments. Yayyyyy.

Migrating Legacy Infrastructure To Kubernetes: Olark’s Lessons From The Trenches

Brandon Dimcheff @ Olark

"Go away or I will replace your devops with a small Kubernetes cluster." YES!

Olark: Live chat company. Copy JS code into your site, now people can live chat. Basically hosted Jabber/XMPP with a neat frontend. Their stack is a lot like ours: lots of Python microservices with RabbitMQ, thrift, Puppet, etc. Currently: hundreds of dedicated VMs at Rackspace. Ooh Ganglia charts.

"Change as few variables at a time as is (reasonably) possible."

  • "The great migration" is the cousin of "the big refactor".
  • Don't try to stand up a full new environment immediately - just the parts you need first.
  • Do a smaller/less critical service first.

"Switching monitoring systems AND hosting at the same time is a recipe for disaster."

  • Perform baseline measurements.
  • linkerd - layer 7 RPC load balancer, supports kube well, thrift and http and so on.

How do you do non-disruptive deploys given long-lived XMPP connections?

  • Hack the (gibson) load balancer to more gracefully switch over.

tl;dr

  • Kube is good
  • Control change (not too fast!)
  • Go with the flow (what it supports)
  • Have a baseline
  • Accept that some stuff is gonna get custom implemented

Ignite Talks (5m each, auto-advance 15s)

Moving from a regular sysadmin role to a DevOps role

Oguz Pastirmaci @ Microsoft

What he did to become devops

  • Learn the vocabulary
  • Get involved in early stages - go to scrum, standup, etc.
  • Start fresh; remember we're all human beings
  • It's all about the people - have to get buyin to do devops
  • Involve security from Day 1, otherwise they clean up our poop

Customer Success: The Missing Link in the Feedback Loop to Engineering

Dylan Schowengerdt @ VictorOps

Customer success is really important, because they will leave your punk ass for a competitor if you don't add value well.

Sometimes several small wins equals a single big win.

Customers will be the first people to tell you what's wrong with your product. You have to filter that feedback, because your developers have feelings too.

Showing customer feedback: Label Ask Visualize Aggregate Highlight (I like this framework.)

Collaborating with Hubot for a More Efficient DevOps Workflow

Mike Nguyen @ GitHub

Otherwise known as "How Hubot lets you be lazy."

Oh look, it's Marley. All over again. There had to be one ChatOps talk in here, didn't there?

You can deploy from chat and monitor from chat. Wonder if the chat bot will scratch my butt for me, too?

Adventures in Public Speaking

Leon Fayer @ OmniTI

(Great advice in this talk!)

  • Impostor Syndrome: You are smarter than you think you are. (Or, the audience is dumber than you think they are.)
  • The audience wants you to succeed: They are on your side! They don't hate you!
  • Everyone has a story: You DO have something to talk about. Inspiration is everywhere.
  • Submit EVERYWHERE. ALL THE TIME.
  • Don't fear failure! You won't get accepted everywhere. You only fail if you don't submit at all.
  • Many formats: lightning, keynote, etc.

Once you're accepted, how do you plan?

  • Depends on you. Whether you memorize or go from outline, etc.
  • Practice in front of people; you won't cut corners that way.
  • Don't be boring! Partially by just being yourself.
  • Do NOT READ YOUR SLIDES. Oh for god's sake yes.
  • Don't overwhelm people. Folks will only have one or two takeaways.
  • On slides: LESS IS MORE. Don't require people to read.

And remember, if I can do this, anyone can do this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment