Generalized Developer Guide

Developer_Guide.md

Software Development: An Overview

Authors: Jake Linenthal and Darwin Corn

Adapted from Jake's writings specific to the MedInsight development team. This is a living document, pull requests are welcome.

The goal of this document is to describe:

• A general philosophy
• Advice for being a good software developer
• Skills that you should have after approximately a year of full-time developer work, depending on field/responsibilities

Communication

A common misconception is that technical skills are the core skill in being a good software engineer. In fact, communication is the most important skill. Communication is not necessarily talking to or interacting with another person. Every time you write code, you are communicating with at least three entities:

1. The computer/execution environment/compiler/interpreter
2. The client who directly or indirectly interacts with the software/feature you are building (this could be an actual person or another process)
3. The programmer who reads your code, either for review or to modify it in the future.

Any piece of communication has four essential aspects:

1. What you intend to say (the idea behind your code in your mind; the idea behind what you want say in your mind)
2. What you actually say (the code; the words you say/write)
3. What the other party hears (the code; what words they hear/read)
4. How the other party interprets what you say (translation to machine operations; the meaning they get out of what you’ve said)

Communicating with computers is annoying because you must be extremely precise. The advantage of this is that any miscommunication is your fault. You have complete control over the first and second aspects of communication.

Communicating with humans is annoying because you usually cannot be precise. The English language and humans in general are inherently ambiguous. This implies that you must leave out or change some information. Therefore, you must take the third and fourth aspects into account and ask yourself, "Will the other party understand what I am trying to say?"

Good communicators have a high probability of deciding whether their idea will be understood by others. Great communicators can take an idea and explain it in different ways to different audiences such that everyone ends up with the same idea in their head. The best communicators in the world can explain an idea in the same way to everyone.

Err on the side of over-precision. Not being understood is usually better than being misunderstood. Three specific pieces of advice:

1. Do not use the words "it," "that," "those," "their," or similar pronouns when communicating. You must be clear about which object you are discussing with minimal chance of ambiguity or confusion.
2. Do not use the word "thing" (see #1).
3. Do not use the word "works.” Use a verb that precisely describes what you expect it to do.

For further information on communication best-practices, read Section L here.

Learning

A healthy software development team has a culture of continuous learning. A software developer cannot possibly know everything necessary to be effective when hired, so it is essential that you be learning at all times. Ignorance is fine and expected. However, make it your goal to shift the object of your ignorance. A great way to bother a senior developer is to be presented with the opportunity to learn about a new concept several times and to not take it.

Second-level ignorance is not fine. Second-level ignorance is when you don't know what you don't know. In a healthy environment, ignorance leads to the conclusion, "I don't know, but I will find out." Second-level ignorance leads to undefined behavior. If you are aware of your ignorance, you will make a decision with this level of certainty in mind. Confidently making a wrong decision when you are second-level ignorant can be dangerous.

How do you find out if you are second-level ignorant? It's hard. Usually you will be presented with evidence that contradicts your understanding of some concept. Ignore this evidence at your peril. It is worthwhile to take the time to ask yourself, "How would I know that this is not true?"

The source code for all of the team's products should be available. Reading other developers' code and commits is a great way to learn and to avoid second-level ignorance. Everyone on the team will be happy to help you understand what is going on/give you context if you ask good questions.

Producing

Learning is only worthwhile if you can put the knowledge gained to use by solving a problem or building a feature. Balancing learning and producing is not an easy, but it is critical to your success.

The goal of any piece of work you do is to:

1. Provide benefit to a customer
2. Make it easier to provide benefit to a customer in the future.

If you cannot identify which category your task falls under and why, then do not work on it. At any point in time, you should be able to explain why the task you are working on will add value.

Getting your code working is an important step, but it is not the goal—the goal is for the piece of work to be used by the intended audience.

We strongly believe that you should not reinvent the wheel. However, you must understand what wheels do and how they work at a high level. Otherwise, you risk putting formula 1 tires on your bicycle or vice versa.

Asking Questions

Contrary to the popular statement, there are both good and bad questions, but asking a bad question is better than being silent.

What is a good question? In general, if you show that you have made an effort to understand and describe the problem or issue, then you will get a positive response. For example:

• Bad:

Why doesn't this work?

• Good:

Why doesn't X result in Y being shown on the screen? I've tried W and I think it should solve the problem because of Z.

Bad questions generally fall into one of the following categories:

• Questions that can be answered with a minimal amount of research and in less than a minute
• Questions that have already been answered, but the asker wasn't listening or paying attention
• Questions asked when the asker has clearly made no effort to define or understand the problem

Although asking a bad question is better than remaining silent, be prepared to respond to feedback and improve your questions in the future. A valuable resource for learning how to ask better questions can be found here.

Problem Solving

Every time you solve a problem, you must be able to communicate:

1. The cause of the problem
2. The scenarios in which the problem will reappear (e.g., when the client has X version of the software and takes action Y)
3. What the solution to the problem is. You must be able to explain this to both the computer and a person. The relationship to #1 should be obvious.

Problem Solving Tips

Begin by asking yourself, “Can I clearly communicate what the problem is?” Close to 100% of issues reported by a customer will be ambiguous. The solution to a surprising number of problems will become obvious if you are able to write down the problem precisely. Some things to keep in mind:

• Every problem has inputs and outputs. Identify them.
• Remove unnecessary details. Try to find the smallest example that will exhibit the behavior.
• Talk out loud. This is sometimes called "Rubber Duck Debugging." It is surprising how much you learn when you vocalize a problem.

Source Control/Automated Builds/Automated Testing

The goal of source control is to be able to see the state of source code at any point in the past.

The goal of automated builds is to create everything necessary to produce a deployable piece of software without any human intervention. Note that this implies that, at a minimum, the feature code, testing code, configuration code, and infrastructure code must reside in source control.

The goal of automated testing is to reduce the probability that there is an error in a deployable piece of software.

Using tools to achieve these goals is not always straightforward, and there are many details, but keep these goals in mind and you won’t stray too far.

Team Meetings

A well-functioning development team requires that everyone contribute in both formal and informal meetings. A well-functioning team also requires that everyone listen respectfully to the technical opinions of others, even if you happen to disagree with said opinion. You are encouraged to challenge the validity of an opinion, but be respectful when doing so.

Although sitting quietly might seem safe, it is a disservice to both you and the other members of your team. There are many examples of teams loaded with bright, hardworking people that fail because the team culture is toxic. Be respectful. For more information on this subject, see this op-ed.

Logistics

Email

• Put the most important point of your email in the first sentence. If you are asking a question, lead with the question. Provide important details immediately following the question.
• Although this fact may surprise some, many people stop reading an email that does not quickly state what is expected from the reader. If you want your email read in its entirety, write in clear and concise language. This is particularly important if you are asking someone to do something.
• Write clear subject lines (see Asking Questions).
• Err on the side of bullet points rather than paragraphs.
• More information here.

Chat

• Be online and responsive. If you don’t want to be bothered, mark yourself as busy.

Calendars

• Accept meeting invites if you are going to attend; decline meeting invites if you aren't.
• Send a meeting invite for PTO so that the team knows when you won't be in the office.

First-year Expectations

Software developers own a myriad of disparate components of a given system or application and some of what follows will need to be learned much more rapidly than the one year mark, while some others may only need the most superficial learning thanks to circumstance. Whether their job title includes the phrase 'full-stack' is irrelevant to the need for every software developer to have generalist-level proficiency at all layers of application development and delivery, lest they encounter the pitfalls of second-level ignorance. Below is a listing of technologies, principles and techniques we expect a developer to be proficient with after a year working full-time.

Security Principles

• Define what constitutes an injection attack and how to harden against it
  • XKCD's Bobby Tables should be a good starting point
• Define what constitutes a Distributed Denial-of-Service attack and how to harden against it
• Understand common goals in any antagonistic action, and common ways to mitigate the risk (the following terms will be useful)
  • privelege escalation
  • denial-of-service
  • SIEM

Development Tools & General Principles

• Source Control
  • Popular branching models
    • OneFlow is my preferred model, more or less; pay close attention to the rebase section
    • While Git Flow has its uses, understand its limitations
  • Difference between git and GitHub
    • a decent starting point
  • Understand why projects and organizations choose a given version control system (hint: it's not always git, nor should it be)
• Automated Testing
  • Unit vs. Integration vs. Acceptance/E2E/Smoke/etc.
  • Understand the utility in something like Netflix's Chaos Monkey
  • Understand the utility and limitation of coverage statistics
• Continuous Integration
  • Understand the interaction between source control and automated testing with respect to continuous integration tooling
• Continuous Deployment
  • Understand the utility of automating the deployment of a given application, particularly in light of what constitutes the goal in Producing.

Object-Oriented Principles

• Define the following:
  • object
  • class
  • struct
  • method
  • property
  • interface
  • inheritance
  • abstraction
  • polymorphism
  • method overriding

Functional-Programming Principles

• Define the following:
  • immutability
  • purity
  • tail recursion
• Define when a purely functional language choice makes sense for a project
  • Explain potential pitfalls of using a purely functional language to develop a greenfield project
• Explain why OO languages like Java & C# have been incorporating more functional-esque features (lambdas & linq, respectively) in recent iterations

General Web & Networking Principles

• Differences in transport protocol
• DNS
• TLS
• Compression
• Understand what CDNs are and the problems they solve

General Database Principles

• Benefits and limitations of ORMs
• Define the following:
  • Indices, differentiate the types of
  • A view vs. a temp table vs. a table
  • Stored Procedures (sprocs)
  • Non-tabular databases and terminology:
    • OLAP/multi-dimensional
      • Dimensions
      • Attributes
      • Measures
      • Grain
    • NoSql/Document-based/Flat databases
      • NoDB datastores like IndexedDb, Gun.eco, etc.
• Excel terminology & defs (oft-underrated by programmers, useful for data munging)
  • V-lookup
  • Pivot table

General OS & Infrastructure Principles

• Be proficient in a shell & its scripting capabilities
  • Some exercises that you should be able to do in a line (or so):
    • Get all files in a directory (both recursively and single-depth) and remove any whitespace from their names
    • Get all files in a directory with a certain file extension, pass them to a binary CLI, and capture the output
      • ex: pass all .docx files to pandoc and convert them to markdown
• Understand buy vs. build, how Microsoft's Windows Server changed that traditional business calculus, and how the public cloud is further expanding on that principle
• Understand when OS-choice for a given software component matters, and when it doesn't
• Understand the 8 layers in the OSI model of software
• Understand basic datacenter construction/architecture
  • Even if your company doesn't manage any part of this, your applications inevitably run on someone's infrastructure that fits this paradigm
  • Terms of interest:
    • SAN
    • NAS
    • SPoF
    • NIC-teaming
    • Bare-metal/Iron
    • Switching (managed and unmanaged)
    • VLANs
    • Hardware Firewall

In Sum

Work hard to get a little bit better every day. Seek precision, clarity, and understanding. If you do, much of this will become second nature. We hope this document proves useful to you in your time as a developer. More than that, we believe that the principles laid out in this document are the basis for an effective career in nearly any organization and role.

Recommended Reading

• The Pragmatic Programmer: From Journeyman to Master
• Out of the Tar Pit
• What every computer science major should know
• High Performance Browser Networking
• Domain Driven Design: Tackling Complexity in the Heart of Software

Fun/Advanced Reading

• The Architecture of Open Source Applications
• Falsehoods Programmers Believe About Names
• Falsehoods programmers believe about time
• Balancing learning and producing is closely related to the problem of "Exploitation vs. Exploration" in reinforcement learning. Unfortunately, there is no easy or universal answer. Wikipedia: Reinforcement learning is a good reference and Exploration and Exploitation in Organizational Learning is an interesting paper.
• Principles of Chaos Engineering