DuckDuckGo is a search engine that's always been known for privacy (among those who'd heard of it). Last year we passed 3 billion annual searches, and it’s critical that we continue to delight users without sacrificing their privacy.
The key, we realized a few years ago, was open source. We created an API and started a community where anyone can join us in improving search results. These contributions are called Instant Answers and they appear above the organic search results.
Our community has steadily grown since then, with hundreds of contributors and counting. Over the years, we’ve learned several powerful lessons about what makes an open source community thrive — particularly one symbiotic with a startup company.
Community was always a part of the mission at DuckDuckGo, even from the earliest days of Gabriel (our founder) in his basement, asking for help and feedback on Hacker News and IRC. Frequently enough, people were suggesting great websites for particular topics that they believed should be ranked first (or just more highly).
Wanting to make use of the valuable insights volunteered to us, we decided to create a way where users could essentially build their own search results.
When we launched DuckDuckHack, the idea was that anyone could improve the search results by using their favorite websites and data sources.
Admittedly, the first Instant Answers were more Easter eggs than anything. Soon enough, however, people began using the search engine to make useful sources of data more accessible. Over time, despite an ambiguous API, our search results gradually became better thanks to open source contributions.
Our lesson was to get inspiration from what’s naturally happening. Where are people naturally offering to contribute? What kind of contributions do people naturally want to make?
Lesson #1: Double down on where your users are already contributing.
As contributions started to come in, we realized that people were contributing for a whole range of reasons:
- Create a feature they personally want
- Gain visibility for another project they're working on
- Build professional experience and credentials
- Work with smart people all over the world
As we made decisions regarding our API, or the open source processes, or communication channels — we kept these motivations in mind during important decisions, and made sure they would still be relevant.
These motivations affected:
- How flexible we were regarding contributions and ideas
- What kind of data sources we accept, and how we credited them
- The communication tools and processes we set up for the community
We want to make sure we don’t inadvertently make a change that dampens motivations — and do as much as we can to fuel them.
Lesson #2: Observe why people contribute, and amplify those motivations as your community grows.
Early on we were just three people managing a search engine with, at the time, a million queries a day. This kept us pretty busy. As the community grew, we found it hard to keep up with pull requests, issues, and quality documentation. After all, open source communities require ownership, leadership, and active curation.
Our mission was, and is, to compete with billion-dollar search giants. However, we wanted to do that without becoming them or emulating their practices.
At the same time, we realized that the most delightful Instant Answers would not come from inside the company. Human knowledge goes way beyond what the best crawlers might ever figure out.
We then understood that community was the key to the search engine’s success. We put the needs of our contributors at the top of our company priorities — and saw contributions shoot up in return.
Lesson #3: Open source communities require attention and investment. Make sure your community is a real priority and aligns with your company's goals.
At first, our collaborations channels were all over the place. We had a Uservoice ideas board, a Zoho forum, a separate docs site, GitHub for code — you get the point. We quickly moved to simplify the community's presence as much as possible — and create clear paths to each task.
To make things easy for community members, there should be a central place for information and collaboration. It should help people decide where to pitch in, how to do that, and how to communicate.
Lesson #4: Centralize your collaboration channels, documentation, and navigation.
It may sound ridiculous, but our original development environment took 2 hours to set up. It required local installation of a custom tool, plus all its dependencies.
We then switched to a cloud-IDE with a ready-to-go environment; it now takes 5 minutes or less. The result? More than twice the number of unique contributors every month since.
The most intimidating part about contributing to open source is getting up to speed. Do everything you can to lower barriers to entry.
Lesson #5: Make logistics as easy as possible for someone new to get to their first commit.
The best source of open source contributors is people who are new to open source. Many developers are looking for that first opportunity to get involved — one where they feel they can make an impact.
A problem we had at first was that it was difficult to complete one’s first Instant Answer. In addition to learning new technology, there were plenty of things to consider, including idea, data source, and design. One of the best investments we made in our open source project was making it easy to add a personal “hello world” Easter egg to our live search engine. This made it easy to just start coding and focus on learning the system. When we did this, we were flooded with first-timer pull requests! Turns out that seeing code run live after just an hour is an incredible motivator.
Another early issue was that contributors could not preview their own work without an employee running a dev server with their new code. As you can imagine, this made development far less exciting.
To solve this, we built a command line tool that allowed anyone to spin up a local server simulating the DuckDuckGo search engine, allowing contributors to immediately visually preview their work. This drastically increased the speed of developing an Instant Answer — as well as made it much more fun.
Lesson #6: Invest time in creating instant gratification: make it as easy as you can for contributors to close a simple loop.
You've made it easy to find, easy to get started, and easy to win — the final step is to help contributors stay involved.
We learned that people stay engaged when they feel others are also engaged. When we respond quickly to pull requests, we found that contributors match our efforts. When we participate actively on the forums, people tend to use them more and engage us more.
If a contribution can't be directly incorporated, try to capture as much value from every contribution, by applying their work somewhere else in the repository. No matter what, make sure to positively reinforce and acknowledge every effort and contribution.
Lesson #7: Every interaction with the community should make people want to stick around.
Our first open source commit took place almost exactly four years ago. Our community has since contributed hundreds of data sources and thousands of pull requests. These amount to around 800 Instant Answers at the time of writing, capable of answering millions of searches.
We certainly have a long way to go, but we've learned that successful open source — like any active community — comes down to providing value for individuals. That comes from being in tune to their motivations, finding ways to align those with your goals, and working hard to get barriers out of their way.
Communities, whether technical or not, pure open source or connected to a company, all come down to people. The more our company invested in helping others, the more we received in turn.