Since early on in the private beta of Documentation, we’ve used a simple, rigid structure:
Tags => Topics => Examples + optional sections
At the time, users wondered if this was the right structure. Since Documentation is a grand experiment, I figured we’d wait to see how the structure gets used and adjust as needed. But rather than being a step in the right direction, tags and topics were just getting in the way. We didn’t address the problem partially because we focused on other broken features and partially because we hoped the current system would pan out. At long last, we’ve got the message: fundamentally the structure we provided isn’t working. This isn’t something we can address by tweaking a few features; it’s the foundation of the entire product. Let’s work together to find a way of organizing Documentation that’s better than what we have now and, perhaps, revolutionizes the entire genre of documentation. If you want to cut to the chase, read down to the “What happens next section?” below.
I’m sure you’ve heard the (misquoted) saying “If you build it, they will come.” As many people over the years have pointed out, it ain’t necessarily so. Plenty of creative people have built stuff that never gets used by anyone not related to the creator. Typically, the moral is that if you want people to use your creation, you gotta build it and sell it. Ultimately people need to have some use for the thing you built or it’ll go unused.
Documentation isn’t a Field of Dreams story. There was no mysterious voice telling us to build it. Instead, after a good deal of success in community-driven Q&A, we scouted out other ways for our community improve upon existing tooling so that developers everywhere can benefit. Having a product that works and that millions of developers depend on every day has one significant drawback: it’s tricky to make structural changes. As Shog9 wrote:
So a few years back, some folks within the company decided to push in a different direction: if we can't experiment on the main Q&A system, let's build new systems, and test ideas that way. Let's throw a whole bunch of pasta against the wall & see what sticks. Will it be mentoring? Videos? Teams? Stories? International sites? Documentation? Dancing?
Kevin Montrose spent a good deal of time looking at what's missing in existing programmer documentation. He proposed the main thing developers struggle with is finding practical and immediately-useful examples. After quite a bit of internal debate, he pitched the idea on Meta. Folks seemed excited about the concept, so we built a system for people to contribute and maintain example-first documentation, and opened it to a private beta. After months of intense work and user testing, we finally were ready to launch.
So we built it and my personal fear was that nobody would come. I need not have worried. In the first ten days, nineteen thousand people tried out Stack Overflow Documentation (beta). Some people got the concept and others didn’t. We fixed lots of (in retrospect) obvious problems and created new ones for ourselves. Despite the predictable drop in participation over the next few months, it felt like we were on the right track. After some initial hiccups, people were able to contribute new content and get it reviewed. Spot checking approved changes showed they measurably improved the quality of topics. Most of topics were not to our community’s exacting standards, but we could see how they might get there.
If this were a movie, the soundtrack would get ominous and there would be gathering clouds in the background. It’s as if we had a cornfield, purchased a bunch of tools and building materials, and invited all comers to make something. At first the sounds of labor are comforting as it means things are getting done. But what, exactly? And can it reasonably be called Documentation? And did we provide the right tools?
We read feedback on Meta and conducted usability tests to find out. The results were . . . inconsistent. You see each tag is its own silo and it’s not easy to navigate from, say, Ruby to Ruby on Rails. And within each tag, topics are their own silos. So people who use tags or topics that are well written and informative will have a different impression than those who have mostly seen examples with glaring problems. The plan was that people would start seeing Documentation via Google searches and propose changes to fix problems. While we are starting to see that happen, it doesn’t seem like it will do enough to produce the exceptional artifacts we’d like to see.
One concern we keep hearing is that the structure of Documentation isn’t working. Some users (both on meta and in interviews) told us our structure was too sterile and generic. Others told us it was too rigid and specific. Nobody's quite sure if we want API-level documentation or tutorials or cookbooks or something else entirely. The optional Syntax and Parameter sections made sense to some and were headscratchers to others. By attempting to serve all sorts of technologies and all types of documentation and all levels of user, we managed to serve none particularly well.
So we are going to try something different. Instead of giving users a blank slate and a fist full of chalk, we’re going to sketch out (with your input) some documentation blueprints. Initially, we’ll prototype what a page of Documentation could potentially look like for just one technology. We’ll test it with Stack Overflow users, our employees, and outside technical writers to get critical feedback. Once we have one artifact that meets our expectations, we’ll see if we can extend the structure to a few more technologies. By starting from the ground up, we hope to find a rich and flexible way to structure Documentation.
If this sounds a little vague and hand-wavy, it’s meant to be. It won’t help to fall into the same trap of completely creating the product and then soliciting feedback. Instead of developing incrementally, we’re shifting to a true iterative approach. To quote Jeff Patton’s Don’t Know What I Want, But I Know How to Get It:
By incremental development, I mean to incrementally add software a time. Each increment adds more software—sorta like adding bricks to a wall. After lots of increments, you’ve got a big wall.
By iterative development I mean that we build something, then evaluate whether it’ll work for us, then we make changes to it. [We're] building expecting to change it. We never expected it to be right. If it was, it’s a happy accident. Because we don’t expect it to be right, we often build the least we have to to then validate whether it was the right thing to build.
I’ve used the two figures above for a number of years now to help illustrate the concept. Artists work iteratively. They often create sketches, decide to create a painting, create an under-painting showing colors and form, then eventually begin finishing the painting. They stop when it’s “good enough” or they run out of time or interest.
In our first draft we’ll create some internet pages documenting parts of Transact-SQL. There are several reasons for this:
-
It’s a technology many of us in the company use on a daily basis and many others are interested in learning. This will give us the widest possible internal audience to test our artifact on.
-
We identified shortcomings in existing T-SQL documentation (both official and from third parties).
-
The Stack Exchange Data Explorer (SEDE) provides an obvious way to demonstrate T-SQL concepts. We also have a wonderful SEDE tutorial which was written by a community member for inspiration.
-
After some debate we figured it didn't matter too much what we picked as long as we picked something.
Part of the exercise is to figure out what sort of structure works for this particular technology. If we do it right, there’s a good chance the structure won’t work so well for, say, JavaScript, .NET, and Git. But hopefully there will be commonalities and broad categories of documentation we can extract and build on.
Yes, please. Since the next phase of development addresses structure, concentrate on writing effective examples. Even with the broken structure and missing elements, people are starting to take notice. One consistently positive thing we’ve heard about the existing system is that people like the focus on examples. It’s difficult to imagine a scenario where Documentation won’t emphasize examples. When we have a better way to arrange content, you’ll be able to move things around.
We’d like to continue hearing from you about what works and what doesn’t for documentation in general and our beta product in particular. I’m not sure we’ve properly expressed our appreciation for months of feedback in private beta, dozens of users we’ve interviewed, and 1500+ meta questions. It’s been immensely valuable, but also overwhelming. So we’re going to start asking for specific feedback that can help us with what we are working on at the moment. To start, please take a look at this companion meta question: What SQL documentation has helped you?
Documentation is very much in beta and it could yet fail. But I gotta say, it’s exciting to be working on something new that has the potential to change how the next generation programmers learn their craft. We don’t know exactly where we are going or what we will find when we get there. No matter what, we’re going to learn something. Come join us on the adventure!