Taken from Medium. I wanted a compact version with all the pieces together, assembled here for my own use. Abridged in some places.
At a high level, we care about three things when considering whether to hire an engineering candidate. Can they build software? Can they learn and teach? Are they aligned with our values?
This is the engineer’s core responsibility to the organisation.
We prefer to give employees problems to solve, rather than solutions. It’s high leverage and allows us to move faster as an organisation. If we hire people who can’t solve problems, we will fail.
We have a healthy sense of urgency. We need the candidate to be able to translate their solutions into code without much friction and quickly grok other people’s code (when working in their language of choice). We want Code Reviews to be focused on the validity of the approach, not basic coding issues.
We want to give our employees ownership. Successful engineers need to be able to unblock themselves, search for answers before asking for help, and make reasonable decisions within their scope without seeking consensus. They should be able to work independently as needed.
Understanding how common data structures and algorithms work — and how to evaluate their trade-offs — are important fundamentals of being able to design good solutions, even if the ability to implement, say, a hash table, is unimportant for the vast majority of roles. Broadly speaking, for most positions we are looking for the appropriate usage of data structures to solve a problem, rather than intricacies or computer science trivia. A more in-depth understanding may be required for certain specialised roles.
Coding in a team is hard, and relies on software being structured in a reasonable way, making use of reusable components, and sensible abstractions. We need engineers who, at a senior level, can design these abstractions and describe how they fit together, and who, at a junior level, understand why these abstractions are important and understand how complex systems are constructed.
Much of a programmer’s job is being told, either by a compiler or during code review, that their code is incorrect or incomplete. It requires perseverance. Building a product to a deadline, where there aren’t always clear answers requires determination. Someone with a growth mindset will see obstacles as challenges, not barriers. Resoluteness is not about tolerating abusive work environments, but about working diligently in the face of hard problems.
Candidates who are eager to learn and capable of doing so will develop faster. Candidates who can teach us something about the world or ourselves will help us become more effective as an organisation.
Asking “why?” and seeking answers are crucial parts of building something new. We want employees who are curious about the world and seek to understand it. Intellectual and emotional curiosity drives creativity.
Employees who are self-aware and aware of their surroundings ask more questions and make more connections. They have more tools at their disposal to solve problems, and can recognise underlying patterns in systems. They are capable of self-reflection, which enables them to grow. They recognise when others require help, support, or feedback, and when someone delivers critical feedback to them, they are capable of being receptive to it.
An empathetic organisation makes it easier to employ the reasonable person principle. Empathy minimises the desire to blame, and helps an employee recognise when someone needs a hand, or deliver feedback in a way that it will be heard. Empathy for users of our product helps ensure we build a platform that solves real needs, and helps us work in such a way that respects the time and energy of our colleagues, by considering them when writing code or designing interfaces.
Communication is key to everything we do at Medium, whether through describing ideas, discussing approaches, presentations, status updates, or a myriad other avenues. The ability to communicate an idea, and to understand communicated ideas is of critical importance to ensure a well-aligned, agile team.
Although we favour individual action and do not seek consensus, the problems we are trying to solve and the vision we are trying to execute are too big for one person to do alone. Every project will involve multiple people, and employees have to be able to work with one another to be effective.
We each have a particular view of the world, but when too many of us have the same perspective, we risk designing incomplete solutions that ignore those who are in our collective blind spot. Each new perspective we can add to the company helps us bridge the gaps between ourselves and the world, and can expose us to ideas that when combined with our own experience, enable us to create something new.
- Build trust
- Do hard things
- Instigate change
- Level up
- Time is precious
- Have fun
There are any number of qualities, competencies and achievements that we could screen a candidate on. However, many of those qualities traditionally prized by tech companies have no causal relationship to the candidate’s future performance. None of the following areas are negative. However, screening for them can cause us to focus on factors that are tangential or outright unimportant to job performance.
A “good” school doesn’t mean the candidate will succeed in their role. The entry practices of many schools are still discriminatory by economic status, race and/or gender. Many excellent engineers did not study computer science in school. Many very accomplished people did not even attend or finish school, cf. Ev.
A high GPA is nice. It shows the candidate was able to work at a high level for an extended period of time. A low GPA in isolation is not a strong signal of anything. The college experience for many requires making compromises based on the practicalities of life. For senior candidates, a GPA is basically meaningless. (I got a higher GPA than Dan, for instance, but… so what?) For junior candidates, project work is more illuminating.
Many companies show excessive positive bias or deference to candidates who have previously worked at major companies like Facebook, Google or Amazon. While these companies are known to have generally good engineers, they are large enough to have bad ones as well, and their hiring practices are such that many qualified candidates are rejected. A long stint at a major company like these can even in some cases be detrimental as many problems are already solved for engineers, a luxury that is not usually available at a startup. Finally, these candidates may also have distorted views on what constitutes “success” for products, be less comfortable with uncertainty, or have views on deployments and the application development lifecycle that are not aligned with the needs of a smaller company. Candidates with large companies like these on their resumes should be interviewed as we would any other candidate.
It is not important for you to feel like you could be friends with a candidate. We are not building a family, we are building a team. A good working relationship, the ability to collaborate, and mutual respect are sufficient. We tend to want to “have a beer” with people who are like ourselves, which can lead to homogenous teams.
Requiring a candidate to have contributed to open source projects is prejudicial. Many candidates do not have the time to contribute to open source projects because they need to work to support themselves or their families. In addition, many open source projects have toxic cultures that are off-putting to many candidates. Finally, it’s not a great filter to kick out everyone who doesn’t want to work on software for free in their off-hours.
It is not important for most candidates to be able to implement a red-black tree from scratch. It is not important for most candidates to know what a Fibonacci Heap is if they’ve never encountered one. It is not important for most candidates to be able to reverse a linked list on the spot. A high level knowledge of time and space complexity, the ability to research, learn and integrate different approaches, and recognition of trade-offs is in most cases sufficient. (In specialized positions, the importance of this might vary.)
It is not a requirement that the candidate know JavaScript or Go. For all but the most specialized roles, interviews should be technology-agnostic. You should instruct the candidate to use the language they are most familiar with.
We want candidates who have the ability to make a decision and back it up with sound reasoning. However, confidence as an isolated trait is not necessarily positive. Many candidates will have difficulty demonstrating confidence in an interview setting (for marginalised candidates, see also stereotype threat). Specifically screening for confidence disproportionately favours white men. Finally, many of the most self-aware people actually appear to lack confidence because they recognise how much they don’t know. The opposite is also true.
Culture fit is a polite way of saying “be like us”. If we’re looking for people who fit in with what we already have, we risk closing ourselves off to new perspectives, and building a homogenous team that suffers from groupthink. Note that culture fit is distinct from alignment of values, and the addition of new perspectives, which are valuable criteria. If someone can #DoHardThings and help us connect to a community who we don’t really understand, they are a potentially valuable resource for us, even if (and perhaps especially if) they make us feel uncomfortable by forcing us to confront our prejudices.
We are trying to build an engineering organisation that is productive, effective and happy. We can only do that by hiring great people. Interviewing is one of the most important responsibilities you have an as engineer; your feedback on candidates will determine how the company evolves.
A few points before you get started:
As you are grading, remember this: we are biased. I’m biased too. That’s ok, as long as we stay vigilant about it, and then adjust. When I’m not thinking about it, I tend to give higher scores to people with a southern British English accent. I associate it with being smart (despite plenty of evidence to the contrary), possibly because I grew up watching BBC News where all the anchors sounded like this.
Interviewing is subjective, but in your written evaluation, try to stick to observable facts. Editorialising on someone’s psyche or value system can be prejudicial, especially if what you write is interpreted by others without you being present to give more nuance.
- Bad: Lillian won’t admit when she doesn’t know something, probably because she’s Latina and I’ve heard that’s characteristic of their culture. I don’t think she’ll be able to admit when she’s wrong.
- Good: Lillian was very reluctant to describe her answer in detail, and talked in generalities instead of writing down the solution. If you have a really strong feeling and are convinced that you have to communicate it, make sure you separate the observable facts from your feelings, and communicate them first.
- Bad: Jackson is a misogynist.
- Better: Even though I was giving the interview, and Peter was just shadowing, every time I asked a question, Jackson gave the response to Peter. Jackson asked Peter clarifying questions, but not me. This frustrated me, and makes me think Jackson would have a problem working productively with a younger woman.
We grade each category on a five point scale: Strong No, No, Mixed, Yes, and Strong Yes. We generally consider Mixed to be more No than Yes. It is the least useful rating you can give, so try to use it sparingly.
It is not essential that you grade a candidate in every category of every section, so you should only grade those categories in which you can make a strong call, based on observable facts. However, if you are only able to confidently grade five or less categories, we should consider the interview format a failure, and think about why we aren’t able to get a strong signal from it.
For each category that you provide a grade, you must provide a note that lists the observable facts that led you to choose that grade. Do not add a grade just to hit some minimum number of categories: if you are unsure about your performance as an interviewer, we can provide additional training. If the interview format does not help you get the signal you need, we can change it.
You will be provided with a sheet that lists the qualities we are looking for, to help you remember what you should be judging the candidate on, and provide space for you to note what you see and hear.
Qualities related to the candidate’s ability to effectively build software.
Independently finding a creative solution is great. Failing to conceive of any solution is a red flag. Only demonstrating a naïve approach is a warning sign. Partial credit for being able to conceive of the solution, but not being able to follow through with it, especially for junior candidates.
Do not make a call if you are not asking the candidate to solve a problem and they are not asked to describe a time where they solved a substantial problem.
- Strong No: The candidate is unable to conceive of any solution to the problem. The candidate is able to describe a basic, or naïve approach, but unable to implement it.
- No: The candidate is unable to arrive at a solution beyond the most basic, naïve approach, even with significant help and hints from the interviewer. The final solution does not account for edge cases. (If due to a lack of time, in the judgement of the interviewer, the candidate would not be able to address those edge cases.)
- Mixed: The candidate conceives of and implements a naïve solution independently, conceives of a better approach with minimal hints, and is able to complete it with help from the interviewer. The candidate dives in to a solution without giving the problem much thought, or asking questions. If a missing edge case is pointed out, they are only able to solve it with significant help.
- Yes: The candidate is able to outline a solution beyond the most naïve, independently, and implement it, covering the vast majority of cases. If one or two hints are required, the candidate understands their importance quickly, and moves on. The candidate asks questions to clarify the scope of the problem and states their assumptions. If an edge case is pointed out to the candidate, they are able to modify their solution independently.
- Strong Yes: The candidate clarifies the scope as a natural first step, is able to articulate multiple approaches and describe the pros and cons of each, makes a sensible decision based on the constraints of the interview format, and is able to fully implement the approach, without any real help from the interviewer. The candidate recognises edge cases and draws attention to them unprompted, then solves them.
Make allowances at your discretion if this is a whiteboard interview, or if they are using tools on a provided computer that they don’t usually use. For coding tests, allow — or instruct — the candidate to use the language they are most familiar with.
Do not make a call if you are not testing a candidate’s ability to produce code, or asking them to read or review code.
- Strong No: The candidate does not know basic language constructs like loops and conditionals. The candidate cannot invoke functions correctly. The candidate is unable to read provided code or describe in any sense what it does.
- No: The candidate is unable to translate their thoughts into code. Little knowledge of the language’s standard library is demonstrated. The candidate chooses nonsensical or non-descriptive variable names when writing code. The candidate is unable to describe the behaviour of any provided code with any precision.
- Mixed: The candidate is able to write basic code, but does not demonstrate strong familiarity with their chosen language’s properties or idioms. Candidate reimplements standard library functions, rather than using what is available. (Do not judge on this if you do not know their chosen language’s idioms or standard library.)
- Yes: The candidate codes fluently and naturally. The candidate chooses to use standard library functions and is able to describe their behaviour when asked. They may use placeholders to abstract away functionality then come back to fill them in. When reading code, the candidate is able to understand the general purpose of the code quickly.
- Strong Yes: The candidate codes without any significant pauses and writes idiomatic code by default. Thought is translated into code without any apparent difficulty. When reading code, and with a little effort, the candidate is able to understand its full behaviour and nuances, and point out more idiomatic approaches where appropriate.
The specifics of this will vary by interview type. For Tech Lead Simulations, we might expect a good candidate to take control, break the problem down and assign them. For a coding interview, a good candidate will work independently, requesting little input except to clarify scope. Generally, we are looking for people who will be able to work with minimal guidance, and people who are able to own problems.
- Strong No: The candidate requires hand-holding and has to be led through every step of the interview. The candidate refuses to make any decisions without consent from the interviewer.
- No: The candidate requires lots of support to complete their task. Decisions are made reluctantly. Approval and validation are sought for each decision. The candidate is unable to describe a time where they led or instigated a project, or if they are more junior, is unable to describe a time when they solved something independently.
- Mixed: The candidate is able to work independently, but seeks approval regularly in a way that abdicates responsibility. The candidate demonstrates some ability to deliver a solution without much input from the interviewer.
- Yes: The candidate is confident owning their decisions and demonstrates a strong ability to work independently. If more junior, the candidate takes ownership of the problem, using the interviewer as a resource to make progress, if needed.
- Strong Yes: The candidate controls the cadence of the task-oriented part of the interview, asking for guidance or clarification only where appropriate. The candidate makes decisions without seeking approval, but describes the rationale to the interviewer.
The specifics of this category will vary by role. For specialised roles like Recommendations Research Scientist, the expectation of algorithmic familiarity is likely to be higher, and we might, for example, have a reasonable expectation of the candidate knowing the naïve Bayesian or collaborative filtering. We would generally expect senior engineers to have a wider exposure to more special-purpose data structures than junior engineers.
Some questions do not require the use of complex data structures. Do not arbitrarily force a discussion about them, unless there is a clear fit for the problem.
To be clear: we are not judging candidates on whether they know terms like O(n). We are judging them on whether they understand that there are trade-offs in building software, and that different algorithms and data structures have different trade-offs.
Candidates who choose and implement appropriate data structures and algorithms without explaining their use may score well here, but more poorly on communication.
- Strong No: The candidate is unfamiliar with the most common data structures, like hash, set, or array. The candidate is completely unable to assess the relative merits of competing algorithm choices.
- No: The candidate has heard of common data structures, but cannot describe which ones are appropriate for the task at hand, or when they might be useful. The candidate cannot explain, for example, the difference between an O(n) solution and an O(n²) solution (the behaviour, not the terms).
- Mixed: The candidate uses hashes, sets and the like appropriately, but cannot, for instance differentiate between a list, a queue, and an array. The candidate is aware that certain data structures are better for certain tasks, but has difficulty articulating why.
- Yes: The candidate understands time and space complexity (even if they don’t use those terms) and can describe the characteristics of common algorithms. The candidate can describe the tradeoffs inherent in different approaches, and can articulate why a specific data structure is appropriate. The candidate can competently implement a recursive algorithm (only if recursion makes sense for the task).
- Strong Yes: The candidate is able to pick an appropriate data structure or algorithm for a task, and has a strong understanding of their relative merits. The candidate’s knowledge goes beyond the basics, and there is a demonstrated familiarity with concepts like heaps, priority queues, tries, or more exotic constructs.
The grading for this portion will depend on the type of interview and the seniority of the candidate. For more senior positions, we would expect sensible code composition as a minimum, and some evidence of larger-scale system design. For junior positions, sensible decomposition of code will typically suffice.
Do not make a call if the task at hand is so simple that nobody would realistically do anything other than write a quick script.
- Strong No: The candidate writes all their code in one function, with no decomposition. The candidate does not see the value in separating out code in their solution, or if the task at hand does not warrant it, in principle. In a design question, the candidate fails to define any reasonable abstractions, or has no idea how to construct the system.
- No: The candidate does not seek to break out their code into reusable components, and doesn’t understand the value of doing so. The candidate is unable to describe the interaction of system components with any precision. The candidate does not demonstrate any understanding of separation of concerns.
- Mixed: The candidate adopts a reactive approach to code or system organisation, thinking only one step ahead at a time. Their finished output has the appearance of being bolted together, rather than being coherently designed. Abstractions exist, but are leaky, rigid or resistant to change.
- Yes: The candidate is able to structure their system in a way that separates concerns appropriately with components that interact through reasonable interfaces. In object-oriented code, the candidate demonstrates a good understanding of SOLID principles (even if they don’t know the specific terms). For simple code, functions are used appropriately to minimise complexity.
- Strong Yes: The candidate is able to break down a complex system into elegantly structured components, and thoroughly describe the interaction model, the interface and the behaviour. Abstractions are clear and clean, given the problem definition, and keep open the possibility of future needs without significant refactoring. Additional thought is given to complex and subtle interactions, like race conditions and idempotency.
Giving up on a particular approach and finding an alternative is fine. Giving up on the entire task because it is too hard is not. This is unrelated to whether the candidate actually solves the problem. When grading on challenges in past positions, exercise your judgement as to whether any further effort was warranted on the candidate’s part — staying in an abusive, hostile or dangerous environment is not a reasonable expectation of the candidate.
Do not make a call if the candidate is able to provide a solution with little effort, and no questions are asked about previous challenges.
- Strong No: The candidate fails to complete the assigned task and is ambivalent about it. The candidate is unable, when asked, to describe a time when they persisted at something, or describes a time when they quit because something was too hard. (Excepting if, in the judgement of the interviewer, the project was unreasonably hard, or the situation was untenable.) Demonstrates no capacity for sticking with something in any prior position.
- No: The candidate shows no strong desire to finish the problem. When confronted with issues in their solution, the candidate shows no strong inclination to get to a fully working answer. Sees difficult times as barriers, rather than challenges.
- Mixed: The candidate professes a desire to finish a task, but shows signs of frustration when issues come up, and becomes negative about the situation. If describing a difficult prior position, the candidate failed to take any steps to make their situation better, did not explore alternatives, and instead moved on in relatively short order.
- Yes: The candidate demonstrates a desire to finish the assigned task, and if presented with issues is suitably motivated to fix them. The candidate describes a time when they persevered with a difficult situation and achieved a good outcome. The candidate describes prior challenges as growth opportunities.
- Strong Yes: The candidate is very determined to finish the task assigned and works hard to do so. If they finish, they are eager to describe next steps, additional work, or extensions. If the allotted time runs out, they express disappointment at not being able to finish, or suggest that they are going to work it out later. For exceptional performance, the candidate emails a solution after the fact (unprompted — do not ask them to do this). The candidate demonstrated incredible staying power in a difficult previous situation, doggedly pursuing their goal beyond what would be expected of an average person.
Qualities related to a candidate’s ability to improve themselves, and make Medium better.
Curiosity does not have to be code- or industry-related! A demonstrated curiosity about, for example (and without limitation), their family genealogy, a foreign culture, cuisine, the mechanics of windmills, literary history, biomechanics etc. are all interesting. Consider as examples within Medium today: typography, the provenance of meat, travel-hacking, and Japanese, among many others.
- Strong No: The candidate demonstrates no interest in Medium, or in the world around them, and provides no evidence that they want to learn more.
- No: The candidate isn’t able to describe any self-directed learning. They accept every statement at face value, without digging in. They ask no questions. (Excepting if the candidate is tired from a long day of interviews and has previously asked questions of others.)
- Mixed: The candidate asks, or has asked, “why?” once or twice, but isn’t particularly interested in the answers, and doesn’t go deeper. The questions the candidate asks are fairly standard, on topics like tech stack, or working hours, without really seeking any deep insight. The candidate has done a little self-directed learning on a topic that interests them.
- Yes: The candidate asks insightful questions about Medium. They ask follow-up questions to go deeper on topics that interest them. They describe times when they have done independent research, on any topic. The candidate describes needing to figure out why something was broken, and derives satisfaction from finding answers. The candidate has done research on the people interviewing them.
- Strong Yes: The candidate displays an insatiable appetite for learning, provides ample evidence of self-improvement and asking questions, and gets excited when they recognise an opportunity for learning. The candidate runs out of time asking questions, and still has more to ask.
We want candidates who know themselves, who will respond well to feedback, and who are self-aware enough to know what their deficiencies are and seek to work with those who can support them.
It will typically be hard to make a call on this in a purely technical interview. However, it is instructive to see how the candidate responds when bugs with their code are pointed out. An inability to recognise or accept a mistake can be problematic. Contrariwise, responding well to a bug with humour and good grace is a good sign.
- Strong No: The candidate demonstrates a significant lack of self-awareness and of introspection. The candidate fails to meaningfully identify any ways in which they could improve. The candidate uncritically gives themselves a 10/10 for performance at all their past positions. The candidate demonstrates no interest in receiving feedback.
- No: When asked to identify opportunities for improvement, the candidate gives false-modest answers like “sometimes I work too hard.” They can’t identify their performance at a previous position relative to their peers. They don’t know how their manager would rate them, or why. They have rejected concrete feedback in the past, and were unable to see why it was offered.
- Mixed: The candidate recognises that they can improve, but speaks in vague terms that demonstrate a lack of significant thought on the matter. The candidate is ambivalent about receiving feedback. The candidate is unable to articulate a time when receiving feedback, or self-analysis, helped them become better.
- Yes: The candidate describes a time when they received critical feedback, and were able to integrate it to become better. They make reflective statements like “I was immature”, or “I wasn’t willing to compromise”, or “I’ve learned that getting buy-in from others is important”. The candidate knows themselves, and has thought about how they can improve. The candidate made deliberate career choices in response to past experiences.
- Strong Yes: The candidate is able to critique their own past performance and identify a number of growth opportunities. They compare their past selves with their current selves unflinchingly, and identify areas where they have improved and the steps they took. The candidate is aware that they make mistakes and demonstrates a desire to learn from them. The candidate speaks objectively about their own shortcomings and describes strategies for mitigating them. They are eager to receive feedback, and demand it.
Judging empathy in an interview can be difficult. Generally, we’re looking for people that understand there are multiple perspectives, are sensitive to other people, and can demonstrate the ability to imagine what someone else might be thinking or feeling. The more senior a candidate is and the more likely they are to lead others, the more important empathy is.
You will generally not be able to make a call in a purely technical interview, if the candidate describes no interactions with others (colleagues, clients etc.). However, there may be some signs like empathy to future maintainers of code — good variable naming, good documentation — as long as the candidate describes the value of those things for that purpose.
- Strong No: The candidate is actively hostile about former colleagues and uses pejorative words, demonstrating a lack of care. The candidate describes people with whom they disagree as, for example, “stupid”. The candidate is incapable of putting themselves in another’s place. They are focussed on themselves.
- No: The candidate makes remarks that imply other people don’t matter, or is dismissive of alternative points of view without due consideration. When describing decisions that they don’t agree with, they are unable or unwilling to speak to any context or constraints the decision-maker faced. The candidate describes situations in terms like “us vs them”, dismissing those with whom there are disagreements as “other”. The candidate doesn’t understand why something upset someone.
- Mixed: The candidate demonstrates a limited ability to understand another’s point of view. The candidate understands that constraints on decision makers exist, but demonstrates little sympathy for people with whom they disagree. The candidate is not particularly worried about the unfair treatment of peers.
- Yes: The candidate declines the opportunity to blame others for a poor outcome, without providing nuance or mitigating circumstances. They demonstrate an intuitive grasp of the reasonable person principle (even if they don’t use the term). The candidate demonstrates that they recognise the impact of their actions on colleagues, and understands that their perspective is not the only valid one. The candidate describes good variable naming and code organisation as a service to colleagues.
- Strong Yes: The candidate does not judge someone’s actions without first understanding their perspective. The candidate shows a balanced assessment of challenges faced at previous roles, and can understand the perspectives of both reports, peers, and managers (as appropriate). The candidate can clearly articulate another’s point of view and describe their worldview. The candidate feels upset when people not within their peer group are treated unfairly. The candidate thinks about how their code will be used, who will look after it, and shows strong consideration for their colleagues in how they build software.
Communication is very broad, and can cover presentation skills, verbal communication, an ability to explain new concepts clearly, written communication, slide decks, data visualisations, gesticulations with hands, listening, asking clarifying questions, and expectation-setting. Communication styles vary widely among candidates; you should focus on whether they are able to get their point across, and able to accurately interpret your questions.
Some people are naturally less communicative than others when speaking to strangers, and especially in interviews. You should assess them on the entirety of their communicative ability, not just their ability to talk fast and off the cuff. It’s fine to coax more out of a person, and/or give them time to warm up. Whenever possible, ask open questions to give the candidate the opportunity to talk.
You should be able to make a call on communication in every interview.
- Strong No: The candidate does not ask any questions, and doesn’t listen. They solve the wrong problem as a result. The candidate is incapable of describing a concept or topic even when they are intimately familiar with it (unless — in truly exceptional cases — the topic is so complicated that the interviewer lacks the context necessary to understand it). The candidate cannot clearly communicate their intentions or thought process, even after significant encouragement and prodding.
- No: The candidate’s intent is frequently unclear. When writing code, the interviewer frequently finds themselves in a position where they cannot in good faith say what the candidate is attempting to do, or why. The candidate cuts the interviewer off frequently. The candidate is reluctant to offer anything more than one or two word answers, or writes in fragments that don’t particularly make sense. The candidate needs a frequent re-framing of a problem to understand it. The cannot adequately explain why they chose the approach they took.
- Mixed: The candidate needs to say things multiple times to get their point across to a good faith listener. The candidate can communicate an idea at a high level, but lacks precision and nuance in the details. The candidate “handwaves” complex parts of code despite being prodded to expound. They are unable to describe a time where they convinced someone of something, or used language to influence something.
- Yes: When solving a problem, the candidate asks questions to clarify needs, and is able to describe an approach at a high-level. They describe how their code works, unprompted. They are able to communicate complex topics to a layperson. The candidate clearly imparts knowledge and concepts through appropriate means, including, but not limited to, diagrams, body movement, writing and speech. When explaining something, the candidate talks at a level appropriate to their audience. The candidate describes a time where their communication influenced a decision.
- Strong Yes: The candidate frequently checks understanding when explaining a complex concept, and if there is any misunderstanding, is able to unpack the problem and effectively communicate the necessary information. The candidate finds the precise word they want to convey the exact meaning they need (even if they takes a few seconds to do so). They are able to explain with great precision why they took the steps they did on a project. The candidate has put together cases that persuade a skeptical audience.
If a candidate has worked exclusively in very small teams, much of this might be theoretical. In technical interviews, judge the candidate on their ability to use you for your knowledge, and willingness to ask for help. Pay attention to how they describe their role on previous teams, and whether they are comfortable sharing credit and accepting blame.
- Strong No: The candidate describes themselves as a loner, doing their best work alone. Demonstrates no understanding of the value of working within teams and cannot describe a time where working with others was beneficial. The candidate is suspicious, generally, of “teamwork” as a concept.
- No: The candidate describes successes as “I”, but describes failures as “we”. The candidate does not like working with others. They do not make use of the interviewer when they are stuck and trying to solve a problem. The candidate fails to solicit input from teammates.
- Mixed: The candidate describes working with others, albeit with some reluctance. They demonstrate a limited capability to work on a problem with others. They ask some questions, though conversation on a problem falls short of a meaningful dialog.
- Yes: The candidate is happy working with others to achieve an outcome. The candidate employs a collaborative approach to problem solving, soliciting feedback, and integrating it into their solution. The candidate brings others with them. The candidate does not merely hand out tasks, and is not satisfied with merely receiving tasks to complete.
- Strong Yes: The candidate demonstrates repeated examples of working with others to achieve a desired outcome. They are eager to work with others. The candidate describes times where other people provided skills that they were lacking. Describes achievements as “we” as appropriate (though taking individual credit for their specific contribution is fine).
While everyone may be (or at least, feel) unique, people from similar backgrounds with similar education and career paths tend to have similar worldviews . Current academic studies in this area suggests that diversity of background and point of view makes companies better. We recognize an inherent tension in this area — that by weighing some demographic backgrounds more heavily, we may be disadvantaging others.
A candidate should not be penalised for the particular demographic that they belong to , or don’t belong to. But we recognise that those who are underrepresented at Medium and in the tech sector more broadly, can offer us points of view that we are currently missing. We should remember that while we are hiring individuals, we are building a team.
Perspective is itself a broad topic — it could be of the world, of the company, of technology, of the product , or something else entirely . As a non-exhaustive example list, life experiences that may lead a person to have an underrepresented perspective may include gender, race, country of origin, languages spoken, military service, sexuality, age, socio-economic class, political preference, education, family makeup, or religion. It is intrusive and illegal to ask direct questions about most of these categories, so don’t. To the same point, asking indirect questions aimed at yielding this information are equally problematic.
But, the underlying motivation — to understand what point of view a candidate might bring to bear on our company our product — is something you should pursue. To this end, open-ended questions may generate discussions on what a candidate may bring to Medium that isn’t in evidence on their resumè. For example, questions like:
-
“What are some of the important experiences that have shaped your view of the world and your work?”
-
“Do you think there’s a perspective that our product is missing that could improve it?”
-
“What are some things you think Medium can do better to broaden our appeal to new users?”
-
Strong No: The candidate offers no new perspectives on the world, the product or the company. The candidate has no thoughts on how Medium can improve. The candidate appears not to have done any research on Medium at all. The candidate doesn’t recognise any of the dangers of groupthink. The candidate describes directly or indirectly homogeneity as the only way to achieve success.
-
No: The candidate fails, when given the opportunity, to describe how they would improve the product, company or engineering systems. The candidate suggests that they prefer to work with people like themselves. The candidate demonstrates no interest in others’ points of view. The candidate offers no new insight, and the interviewer learns nothing new in the interview, no matter how small.
-
Mixed: The candidate has done a little research on Medium, but knows little more than that it’s a blogging platform, and that Ev is the founder. The candidate solves a problem in the same way that the vast majority of candidates do.
-
Yes: The candidate is able to describe the needs of a demographic for which Medium does not yet adequately cater. The candidate is able to teach the interviewer something about people’s perspective of Medium, and recognises the value of diverse perspectives. The candidate is able to contextualise a problem or solution in a way that the interviewer hasn’t considered. The candidate teaches the interviewer something they didn’t know. The candidate adds something unique to the company that we don’t already have.
-
Strong Yes: The candidate has extensively researched Medium and comes prepared with lots of ways the product could be improved. The candidate has strong ties to a community that is underserved by Medium, or with whom Medium currently has difficulty communicating. The candidate has ideas of how Medium could be made more useful or opened up to a new audience by serving a specific set of needs that haven’t been identified within the company. The candidate identifies an edge-case in a long-standing interview question, of which the interviewer was previously unaware, or solves the problem (correctly) in a way that hasn’t been seen before by the company. The candidate imparts substantial wisdom or something revelatory to the interviewer.
Qualities that ensure the candidate is aligned with Medium.
Trust is central to all we do. It allows us to distribute authority, work in parallel and treat each other with respect. Backchannels are particularly useful for understanding whether a candidate has previously built trust. Candidates who have undermined colleagues, or been economical with the truth in previous positions are unlikely to reveal that about themselves in an interview.
- Strong No: The candidate refuses to delegate work to others. The candidate is duplicitous, dishonest or disingenuous. The candidate has actively undermined colleagues. The candidate prefers to achieve results through fear. The candidate feels the need to check every piece of work and have a say in every decision, and is unable to temper that need. Nobody believes any deadlines that the candidate sets.
- No: The candidate doesn’t trust others to do their job, and demonstrates micro-managerial tendencies. The candidate has been on performance improvement plans at multiple previous companies for the same deficiency. The candidate has a history of missing deadlines or being late, and has taken no corrective action to improve this. People are unwilling to work with the candidate again.
- Mixed: The candidate has previously been unreliable, but has made a good faith effort to improve it. When describing a time when they failed at something, the candidate is unwilling to take responsibility for their part. The candidate has failed to update stakeholders on a number of occasions, without adequate justification.
- Yes: The candidate has earned trust from peers and leaders in previous positions. The candidate has been entrusted with delivering important projects, and has validated that trust. The candidate communicates effectively with stakeholders, hitting deadlines or giving good notice why they won’t be met. The candidate is honest and straightforward in their communication, and does not dissemble or sandbag. The candidate delegates effectively.
- Strong Yes: There are multiple references who state their unequivocal trust in the candidate. The candidate tells the truth, respectfully, even when it is hard, and is willing to speak truth to power. The candidate has delivered multiple projects on time and to budget, and been given ever more critical work throughout their career as a result. The candidate gives out problems, not solutions, and encourages and empowers subordinates to work autonomously, as appropriate. The candidate gives people the benefit of the doubt when they make a new mistake.
We want people who relish a challenge.
Do not make a call in a technical interview if the interview question is not sufficiently challenging. If making a call, consider how they reacted to a hard problem that they haven’t seen before.
- Strong No: The candidate has chosen positions that were easy, because they were easy (except if in the judgement of the interviewer they were deliberately taking a break after a stressful position). The candidate shies away from difficult work.
- No: The candidate has stayed in a non-challenging position for an extended period of time and demonstrated no desire to challenge themselves. The candidate is reluctant to attempt something at which they might fail.
- Mixed: The candidate has worked on some difficult projects, but prefers to follow, rather than lead. They have reluctantly worked on some challenging projects, but only at the insistence of others, and did not demonstrate an appetite for it.
- Yes: The candidate has willingly taken on tough challenges that involved some risk. The candidate has said yes to projects where they didn’t have a clear idea of how to achieve success, and achieved success anyway. The candidate has solved complex problems and developed innovative solutions. They relish challenges and often take on hard and unglamorous work because it is important and necessary.
- Strong Yes: The candidate has worked on incredibly difficult projects, inventing wholly new ways to solve problems, whether in code, process, or business model. The candidate is recognised by their industry peers as being innovative or a leader. The candidate has executed turnarounds on stalled or failing projects. The candidate has achieved success despite structural barriers to that success within society or a company. The candidate has a demonstrated history of leading the way for others to follow.
Instigating change can come in many forms — advocating for a new product, working to change the demographic makeup of a company, defining a new business model, pushing the company out of complacency. Engineers can typically instigate change by writing code, but organising meetings, getting decision makers interested in a problem, or writing op-eds or company memos can also count.
- Strong No: The candidate is satisfied with the status quo, and actively avoids or resists change. The candidate only changes when forced to by externalities. The candidate feels no urgency to improve process or solve problems.
- No: The candidate allows other people to lead on change, reluctantly going along with it only when necessary or expedient. The candidate is a passive observer, rather than an active participant. The candidate can solve a problem when they are given it, but don’t seek out opportunities to do so.
- Mixed: The candidate participates in change and happily goes along with it, but they are not responsible for it themselves. The candidate is flexible enough to deal with the effects of change in an organisation, but does not particularly take much initiative. The candidate is able to describe a rare case of them causing a change to occur.
- Yes: The candidate has taken initiative to solve problems and improve processes. The candidate can describe the issue, what they did to address it, who they involved, and the value of the change. The candidate volunteers to solve issues. The candidate has written or spoken extensively on a topic with the aim of influencing decision makers.
- Strong Yes: The candidate repeatedly drives change in their organisation, above and beyond what might be expected of someone in their role or seniority. They have caused large numbers of people to change in a meaningful way. They have helped influence the direction of a product by prototyping without seeking formal approval. Multiple product features exist because the candidate actively lobbied for and/or implemented them. The candidate sees driving change as a default behaviour, and does it unconsciously.
Can the candidate make things or themselves better?
In a technical interview, is the candidate satisfied with the brute force solution? Or do they look for a better approach without being asked? It’s ok if they ask for hints on how things can be better — in this section we’re mostly concerned with “are they satisfied?”. Ideally, they’ll never be satisfied, given the constraints of the interview format, unless there is a “perfect” answer and they find it. In a non-technical interview, does the candidate strive to improve themselves? Do they research things or learn new skills?
- Strong No: The candidate is satisfied with their current capabilities, and does not want to add additional skills. The candidate considers their learning “done”.
- No: The candidate does not take an active role in improving themselves. The candidate has not progressed in any meaningful way for a long time (relatively), and their career has plateaued. The candidate does not demonstrate the ability to learn from their mistakes.
- Mixed: The candidate has made a few half-hearted attempts to improve themselves, though not in any consistent way. Their career has been stop-start, with long periods that demonstrate no progress.
- Yes: The candidate demonstrates a strong desire to improve their craft. They deliberately work outside their comfort zone in order to broaden or deepen their skills. They are continually learning, whether through courses, reading material, practice or another method. The candidate has advanced steadily in their career.
- Strong Yes: The candidate is never satisfied with their competency or capabilities, and is constantly trying to improve. The candidate can describe multiple areas where they are aiming to get better, or skills they would like to deepen. The candidate has an exceptionally steep career trajectory, advancing quickly in seniority and experience through dedication to self-improvement.
Does the candidate respect people’s time, and their own time? Can they effectively prioritise things so the most important work is done first? It does not matter, within reason, what time an engineer is at their desk or in the office. What matters is their ability to commit to a deadline and then hit it.
In a coding interview, do they have the urgency necessary to get to the best solution they can in the limited time they have? For junior engineers, project work or internships with a deliverable are also valuable indicators.
- Strong No: The candidate has no sense of urgency and is content to bide time. They are content to do average work that does not satisfy them, so as to make rent (unless, at the interviewer’s discretion, there are extenuating life circumstances that require them to do so). The candidate has no demonstrated history of hitting a deadline. When leading projects, they have no concept of work-life balance for their subordinates.
- No: The candidate works hard, but is unproductive or inefficient. The candidate does not demonstrate any understanding of time-boxing, and has spent large amounts of time on futile endeavours (except if, in the judgement of the interviewer, they were trying something truly new and innovative). The candidate sees no value in deadlines as someone who is doing the implementing. If leading a project, the candidate cannot hit tight deadlines. The candidate is frequently late.
- Mixed: The candidate recognises the need for deadlines, but is content with setting or meeting conservative timelines that allow them to take their time. The candidate is sometimes late to meetings with no prior warning. When writing code, the candidate does not manage to finish the problem in the allotted time when, in the judgement of the interviewer, they could have if they had worked a little faster.
- Yes: The candidate demonstrates a healthy sense of urgency. They are able to prioritise competing needs effectively. The candidate is able to balance ideology with pragmatism, and is cognisant of time constraints. The candidate rejects timelines that are incredibly conservative as wasteful and unambitious. The candidate is always on time, or provides timely updates so as to avoid wasting others’ time. They try to front-load their work, so as to have time to deal with unexpected consequences later.
- Strong Yes: The candidate has a strong bias to moving fast, shipping products and improving on them. The candidate is very respectful of other people’s time. They do not schedule needless meetings. They set aggressive but reasonable deadlines and then work hard to meet them. The candidate is constantly trying to find ways to be more efficient, and drives the organisation to do the same.
Medium is not a family, even if many of us are very close. We don’t apply the airport test. But we do want a happy environment that people enjoy working in every day. The candidate doesn’t have to be a comedian, or extroverted, or the constant centre of attention, but they should enjoy their work, and demonstrate an ability to have fun. Your sense of fun and theirs doesn’t have to agree. Be careful not to conflate this with “able to take a joke at their own expense”. That’s not expected of candidates. Likewise, the candidate is not expected to be good humoured about the structural inequalities in our industry or society.
Candidates are not expected to be happy all the time. Much of the work we do is demanding and often frustrating. However, there should be some demonstrated joy in the craft, or examples of when they enjoyed their work. An excessively negative personality can be toxic to a small team and drain it of energy.
- Strong No: Makes jokes about marginalised groups, or laughs maliciously at others’ misfortune. Sees work as a place to clock-in, do their eight hours, and clock out. Shows no interest in learning anything about their colleagues. Is excessively negative about the business and/or the product.
- No: The candidate can’t recall a time when they enjoyed themselves at work. The candidate is generally pessimistic and unable to describe many or any highlights of any previous roles. They appear to take no satisfaction in their work.
- Mixed: The candidate is unable to see the humour in anything they work on, and demonstrates no real love for it. While not pessimistic, they are not able to generate much enthusiasm for the things they’ve worked on, or for Medium. Nothing really excites them.
- Yes: The candidate brings a sense of humour and joy to their work. The candidate is able to laugh at themselves (when they are the ones doing the joking) or some of the absurdity of the job. They are generally excited about their opportunities and are upbeat and positive.
- Strong Yes: The candidate derives immense pleasure from their work. They have an easy, affable nature that puts people at ease. The candidate is optimistic and tries to look on the positive side. For more introverted candidates, they are able to describe something they were very proud to have shipped that they enjoyed working on.
You will be asked to give a final overall assessment of the candidate, Strong No, No, Yes, or Strong Yes.
We can teach someone who is lacking in one or two technical categories, provided they have the right attitude. It is harder to change a person’s personality, or gain alignment with them on values.
Below are general guidelines for how to think about each grade. There may be exceptions to this, at your discretion. Each case is unique, so exercise your best judgement. Although we are aiming to grow quickly, it is still true that a bad hire can have far-reaching and wide-ranging consequences. You should consider that your reputation as an interviewer is attached to your recommendation.
Give an overall Strong No only if you think hiring the candidate would cause significant harm to Medium. A single overall Strong No from an interviewer is likely to result in rejection of the candidate in all but the most exceptional circumstances, even if other interviewers are in favour. Be judicious in your use of this grade.
Give a No if you think the candidate isn’t right for Medium. Generally speaking, a Strong No in any of the personality or values categories will indicate at best an overall No, even if there are Strong Yes results for technical competencies. An overall No is also appropriate if the candidate has technical deficiencies that are too great to correct with reasonable on the job training. This will be typified by one or more Strong No results, or mostly No results in technical categories.
Give a Yes if you think we should hire the candidate. Generally speaking, majority Yes and Strong Yes for personality and values categories may indicate an overall Yes, even if there are one or two technical No grades.
An overall Strong Yes is a strong signal, and should be reserved for someone who you think we really need to have on our team. Be judicious in your use of this grade.