What qualities make for a good startup hire in general and what are the special challenges of the first hire for the team, or, the foundational member of any new team. This really applies pretty broadly to any type of team, not necessarily ops in particular.
So if you're a founder, and you're saying, "I need an ops team. I'm going to sit back and think about the ideal operations engineer and the list of skills and expertise that I would really like them to possess," and your list probably looks something like this.
Obviously, you want them to be an expert at Ruby or Python, some deep systems knowledge is great. They should also know how to recompile kernel modules. It would be super awesome if they were good at security, and they know how to do TripWire and all that shit.
They should be a great networking engineer. It would be really great if they know how to sniff TCP packets and unpack them, obviously they need to know Chef and Puppet. All this shit.
Oh, you know what else would be awesome? If they only want to work for $25,000 a year plus stock and snacks and 0.01 percent equity. So, you want a unicorn? Well, we all want unicorns, but we don't get them.
What you do get, hopefully, is an engineer. An individual with very specific strengths and weaknesses. A background that may or may not be relevant to what you really need.
This means that you need to get really ruthless about narrowing down the list of skills. Not even skills, but the list of strengths that will cause your company to succeed. I'll talk a little bit more about how to identify your strengths and how that fits in a little bit later.
I will say this. In my experience, for startup engineers across the board, one of the strongest predictors of success is something I've heard referred to as a T-shaped engineer.
A T-shaped engineer meaning they are broadly literate across technology, like a range of technical topics. Ops is so broad, you could be a specialty in network engineering, routing, operating systems, and a million different kinds of databases. You could be a caching engineer.
They should at least be able to go and speak intelligently across a stack and they should have demonstrated the ability to go deep into at least one area.
When Parse was interviewing me, I had literally never used any of their core technologies. I had never used AWS. I had never used Ruby, Chef, Mongo, Redis, Cassandra, the list just goes on. I had never used any of this shit.
I had experience scaling really fast and I had experience in mobile, several times, with 10X companies and that was what they wanted. They were very clear on that point and so they didn't get distracted by things like, "You don't actually know how to solve our Mongo problems when you're coming in the door. They just assumed that I would learn that shit, and they were right.
By and large, ops people are really good at picking up whatever it is that they need to know.
In my experience, there are a few qualities that all good ops engineers have in common. They may not be the best software engineers and they might not write the best code the fastest, but they are allergic to doing the same thing twice, and they have sufficient coding skills to make that happen.
All good ops engineers feel really, personally responsible when their stuff breaks. Most good ops engineers have a really hard time taking vacations. Great operations engineers, and again, I'm going to extend to great engineers period, have really strong opinions on a wide range of technical topics, but they're not dogmatic about them. They're not religious about them.
They're aware that every engineering decision involves countless trade-offs, and they're persuadable. Emacs versus Vim is different, we'll just leave that aside. It's totally reasonable to be religious about your editor.
All great ops engineers really strive to simplify their architecture. They want to maintain the fewest things possible. They strive for reusable solutions. They strive to reduce complexity and they will push back whenever you want to add a new element to the stack, just by reflex.
It's a negotiation. I think about ops people the way I think about good lawyers. It's a good lawyer's job to never say, "No." It's to tell you how to get to "Yes." Maybe you don't like that answer. Maybe you're not willing to make those trade-offs, but a good operations engineer is there to help you get to where you want to go.
Ops engineers value process and the reason why that is, is not because anyone loves filing tasks, or attending SeV reviews, but it's because process is what prevents you from making the same mistake over, and over, and over again.
Empathy is the thing that the dev ops movement is really about. It's not about hiring dev ops engineers, because dev ops engineers are not a fucking thing. It's about managing the constant tension between the need for software engineers and product people to ship code, to get features out there to help the company move forward and succeed, and the need of the infrastructure people to make sure that it happens safely and doesn't take anything down.
Dev ops means you have shared responsibility for both goals. It means ops people are saying, "Yeah, yeah, I get it. We need to ship stuff as fast as possible. Let me help you figure out how to make that happen." The software engineers are like, "Yeah, we don't want to get paged in the middle of the night either, so let's figure out how to make that happen."
I will also just say, anecdotally, every good ops engineer I know, is a fucking wizard at Bash. Otherwise, it would just vary, but everyone knows Bash.
What's not on this list? Things that do not predict great startup ops engineers. They're great at whiteboarding code or any specific technology or language.
I'm also going to say this, I want to talk about the big company pedigree issue. VCs love big company pedigrees. They love it when you hire someone from Google or Facebook. The people who come from Google or Facebook have definitely demonstrated some superior skills at certain things. They have demonstrated their ability to work in very large, very structured environments on a problem that is a slice of a slice, of a slice, of a slice, of a slice, of a slice of a problem.
I think this is a trap. You're selecting for a different set of criteria when you're hiring for Google and Facebook than you are when you're hiring for a startup. Some engineers can cross that gap, and really shine in both areas. But they are rare. Most of us gravitate towards one or the other and can function okay in the other environment.
If your engineer from a big company has been at that big company for a very long time, especially if they went to that big company straight out of college, they don't live in the real world, in technological terms. They can learn, but they're starting from a place of being pretty far behind.
What does the real world use for a web server? Or for a load balancer? Or for a database? Or for a caching solution? Or for development tools? Or for continuous integration? I know an amazing SRE at Google who has said, "What's a LAMP stack?"
I'm not trying to say they can't learn, they can. But I feel like it's about the same amount of risk hiring someone straight out of a long tenure at a big company and hiring your first remote employee. It's a big cultural shift. They don't know if they're going to like it. You don't know if you're going to like it.
There's also a kind of learned helplessness that can set in at big companies after a certain amount of time. This is basically how I feel about the characteristics that make you succeed at a big company or a small company.
People who really, really love working at startups are often the kinds of people who will implement 80 percent of a solution and then move on to the next most important thing. That's usually what you want at a startup. At a big company, that's not really looked upon as a good thing.
Ops at a startup is fundamentally a very reactive role. Every role at a startup is a fundamentally very reactive role. Things are changing behind you all the time, but ops especially. At a big company that's really doing it right, it's not.
Now let's talk about if this is this your very first hire for the team. The founding team member for any new team is incredibly important, because it sets the stage. It sets the foundation and it sets the culture for all of your subsequent hires. You're going to rely disproportionately on their judgement to tell you if anyone else if your hire is doing well or not.
You're going to rely on their judgement to help you interview and hire anyone else for that team. If you get this right, you can substantially delegate a whole bunch of technical decision making to someone who can do it better than you can. If you get this wrong, you will have a messy clean up job on your hand.
So your first hire is your unicorn. Technical judgement is incredibly important. This person is going to be making decisions for your company that will be with your company until literally the day it dies.
At Linden they are still using things that the other founding ops engineer and I were embarrassed about hacking together 10 years ago. It's still there. They don't have to do everything perfectly, but it's important that your first hire makes more good decisions than bad decisions.
Your ops engineers have to have enough credibility with your software engineers that they will listen to them and so your ops team can actually have the power and the credibility to get shit done.
Your life will also be so much easier if your first hire is fairly senior and can grow into that tech lead role or management role.
It will be very awkward if they aren't capable of this and you have to hire or grow someone to manage them later. Great operations engineers are great communicators, full stop. They're good at getting everyone on board with the vision for the team. This is what needs to happen. This is what we need to pivot from building features to working on reliability. They have to have that credibility. They have to know how to communicate with the other teams, and let me just say, if you find these qualities in someone, it's worth paying for them.
Senior people who are worth their salt should know what they're worth. If you're hiring software engineer number 25 at the same time as you're hiring what you want to be you