author: @sleepyfox
title: Expertise and interviews
date: 23-Apr-2023
In 2001 Paul Graham, founder of Startup Incubator Y Combinator wrote a blog post about LISP called 'Beating the averages'. I don't want to talk about LISP, or startups, not today anyway.
In the post he described how a hypothetical programmer, of a hypothetical mid-powered language called 'Blub' viewed other languages:
As long as our hypothetical Blub programmer is looking down the power continuum, he knows he's looking down. Languages less powerful than Blub are obviously less powerful, because they're missing some feature he's used to. But when our hypothetical Blub programmer looks in the other direction, up the power continuum, he doesn't realise he's looking up. What he sees are merely weird languages. He probably considers them about equivalent in power to Blub, but with all this other hairy stuff thrown in as well. Blub is good enough for him, because he thinks in Blub.
Now, to interviews: let me explain. No, there is too much, let me sum up.
The first problem with interviews is that they are very lossy measuring devices, there is signal, but there is so much more noise that it is very difficult to determine what large scale artefacts you see in the data are just noise, or whether they are buried signal.
The second problem with interviews is that the measuring device is subjective, biased, and imprecise. That would be you. Also me, and everyone else, but you're the one reading this.
Most executives believe that the big issue with hiring is finding prospective candidates (it isn't), or secondarily hiring people who can't do the job (well) - a problem known as the false positive. Whilst the false positive is in deed an issue, it can be mitigated against - and can also be (fairly) easily detected post-hiring, even if some damage will have been done.
Far more insidious is the false negative, the person who would have been an excellent hire, but you turned down because X. Where X is reasons that have more to do with implicit (or explicit) bias, and nothing to do with actual performance. The false negative is more insidious because you will never know, the person doesn't join your organisation, you won't be aware of their outstanding performance in their next job/organisation, and you will tell yourself that you dodged a bullet and that it was a good thing that you didn't hire that person, assured in your self-righteousness.
When we look at someone who is junior to us in some skill or practice, we can determine this relatively easily. Mostly. They will make mistakes that we recognise from our own development, or the development of juniors that we coach. Our determinant of whether someone is equal to us is that there is an absence of such mistakes. But how do we tell when someone's skill or ability in a thing is superior to our own? This isn't a Chess match where being beaten is a clear indicator of skill. Of course we might attribute it to luck. But being consistently beaten paints a more convincing picture that they are operating on a higher level. We might presume that when we talk to someone that it would be obvious that they are more expert, that the candidate would just dazzle us with their brilliance.
The reality is that this isn't the way that superior skill manifests, except to juniors. Instead the expert operator does things differently to the merely competent operator, they use techniques or practices that seem odd, unusual, not the choice that we would have made. They obsess over things that we don't value, that seem unimportant or seem like nit-picking. They may take time on things that we wouldn't bother with, because we're 'pragmatic' or 'delivery-focused'. They see things that we don't, but because we don't value those things we view their perception as being distracted by insignificant details. Often when we query them over these things they aren't able to provide cogent explanations over why they do this thing, why it is so important, and what makes it a better choice then the thing that we might otherwise do, because tacit knowledge is by its very nature implicit and hard to describe.
This is why if you are interviewing for e.g. a staff engineer, you need a senior staff engineer or higher to at least be a part of the interviewing team, if not running it.
This is why for senior operators, whatever the discipline, interviewing is possibly the single most important thing that they can be doing for your organisation.
This is a hard thing to accept, because so many people will be wanting you to DO things, build things, make decisions - important decisions! Guide people, mentor people, lead teams, make change. Don't get me wrong, these are all important things, but they are also less important than hiring, because hiring is a multiplicative, compounding advantage, whereas doing things is only additive.
ABH - Always Be Hiring.