Development, like science, is a messy endeavor. It’s near impossible to control all the variables. Testing code is a noble goal, but it’s very easy to test the wrong thing. Many experienced scientists are fooled by their senses and biases. To account for that they crafted and refined a Scientific Method. That method has been stress tested by centuries of experimentation, discovery and peer-review. It probably wouldn’t hurt to try to apply it to development and see what it can offer.
- Inquire
- Hypothesize
- Disprove
- Reflect
- Publish
- Think Different
Let your mind loose of the tight corset of implementation. It doesn't mean you have to forget reality. The goal is to solve real problems. Observing the world around should prompt you to wonder: "What if it worked better?".
Explain in how you could solve a problem. "To make X better we could do Y". A hypothesis is not a test, all you need do is formulate a possible solution to a problem.
In Science, you don't prove a hypothesis is correct, you prove it isn't incorrect. Learn how to falsify your hypothesis and you will become a better tester first, and a better developer in the long run.
What happened? Did your hypothesis hold up to reality? Or did it blow up in a thick dust of incorrect assumptions? This is a crucial phase where you have to be honest with yourself. You have to be curious enough to poke the very corpse of your beautiful idea. This is rarely a fun thing to do but it can become fun once your realize how much you learn here.
This is an odd one, it doesn’t really fit our idea of software development. Does it? But what if, once in a while — when one of your assumptions is shot through experimentation — what if you wrote something about that publicly. Allowing yourself to reflect on your process, letting peers get a peak at it and maybe learn from it, and allow you to document your progress.
Resilience matters here. At the core, developers — like designers — are people who like to make things. Sadly, the solution of the problem we’re trying to solve is not always the one we imagined. It often takes time, error, and iteration for us to find a truly elegant solution. The child-like ability for untammed imagination is what sets apart the people who will try once or twice and give up, from the people who will relentlessly strive to improve their work.
It’s about craftsmanship, not talent.
Eric had good feedback on experimental setup. It could be interesting in the section on experimentation to teach developers about clinical trial techniques and dangers: bias, double-blinding, controls, contamination, margin of error, statistical significance.