Create a gist now

Instantly share code, notes, and snippets.

What would you like to do?
Healthy Open Source Projects Need People (Redux)

Last month, I wrote a post for the Engine Yard blog called Healthy Open Source Projects Need People. If you haven't read that, you should go and read it now.

I'll try not to repeat too much of it here. But I was very pleasantly surprised by the reaction to it, and thought I'd do a bit of a redux to summarise the conversation around it.

The basic premise of my post was that a healthy community is more important than a working product. A concept that some people find alien. And something which took me many years to realise. It was really only through my experience on the Apache CouchDB project that I learnt this lesson. And in part, through absorbing the Apache Software Foundation's ethos around community over code.

Why is it more important to have a healthy community than a working product? Because with a healthy community, you can fix the product. In fact, with a healthy community, you can fix everything. It is the most important thing to get right.

There was an interesting discussion on Hacker News about my post.

User mattgreenrocks made the argument that some projects do in fact reach feature completion, and so community is not so important in those instances.

In response, I argued that feature completion isn't enough. Software literally rots. (Okay, not literally.) You need to keep it up-to-date if you want it to continue working.

It is possible to reach feature-completeness. TeX is probably the most well known example of this. It was feature-frozen at 3.0, and the only changes made now are bugfixes> The problem arrises when the things your program depends on go away. For example, if you have a project that is written for Ruby 1.8.7, after June of this year, that version will receive no more updates. If there's some particular feature of 1.8.7 that you depend on, or some bug found, or some security vulnerability discovered, users might eventually find themselves in a spot of bother.

The web of dependencies for most projects is bewilderingly complex. The stacks that we write for are moving targets, and unless your software is moving to keep up, eventually there's a good chance that they just wont run any more.

This sort of thinking is why the IETF publishes Internet Drafts as ASCII-only text files. They've limited the dependency chain to ASCII, because they are hoping that isn't going away any time soon.

User saurik brought up ncurses as a counter example. His argument was that some projects are slow moving enough to not need constant maintenance. I thought this sounded like a good argument, so I decided to investigate the project to see what I could learn.

The most important thing I wanted to communicate was that active contributors are very important for a healthy project. And that if you add people to a project, the project stands a chance of outliving your individual contributions to it.

If a project required constant fixing because, say, the code was very bad, then yes, I agree that would be a bad sign. But I was primarily thinking of dependency maintenance.

I took a look at ncurses to see if I could learn anything about their success.

I count 18 contributors in the README file, which is exactly the sort of thing I was thinking about really. Seems like the package has been passed from one maintainer to the next as people's situation changes. This sort of thing is important! If ncurses had met with the same fate as most GitHub repositories, nobody would use it any more. It would be forgotten about.

I took a look through the file too, and it looks like ncurses has very few dependencies beyond requiring a sane build environment. And I see that ncurses releases once every few years, and has been doing since the 90s. Oh, and the mailing list seems alive and well.

All in all, ncurses seems to be exactly the sort of healthy project I was thinking of. Regular contributions, shared maintenance, very few dependencies, and a predictable release cadence.

User coldtea mentioned that many big OSS projects have commercial backing. And user dreen pointed out that this can be risky, because if the money dries up, the project will too. This is a great point. But there are ways to mitigate against it.

Indeed. Commercial contributions are very valuable, but it's important to take steps to reduce the inherent risk. If AcmeCorp stops paying its employees to contribute to your project, do you still have a project left? If that thought scares you, it's a good indication that you need to increase contributor diversity. Perhaps that means more volunteer contributors, or perhaps it means additional corporate contributions. But do not put all your eggs in one basket!

In the comments on the original blog post, commenter Steve cited the oft occurring problem of maintainers running out of energy as the project grows. This is a very common problem. In fact, there's a great talk by Fat called What Is Open Source & Why Do I Feel So Guilty? In this talk (which I highly recommend you watch) he tells the story of one of his OSS projects, and how over time, he found himself spending most of his free time doing ticket triage and management tasks. And he talks about how damaging this was. Which is something I can completely empathise with.

But at the same time, I think that these issues can be avoided if you start recruiting contributors early and often. As you grow, it becomes easier to recruit. And the more you recruit, the more help you will get. And conversely, the more free time you retain. This is, at it's heart, about sustainability in project growth. The myth of the super productive hero is a damaging trope.

And as I commented a post by Navin Ipe:

There's a mindset that seems to come from the isolationism of the great hackers of yore, who would sleep under their desks or build hardware in their garages. They did it alone, so can't we always go it alone, like pioneers? But it's just a myth. Even the golden age hackers worked together. In fact, you could argue that they were more social, more engaged in communities, than these renegades are now.

If you attempt to do everything yourself, you're missing out on the network effect. You're missing out on the one of the primary benefits of the internet and our new ways of working together. Something that Clay Shirky covers in more detail in his book, Here Comes Everybody: The Power of Organizing Without Organizations.

As I said to Dave:

Yes, this is a common problem. Adding contributors early, and adding them often can help to alleviate maintainer stress as the project becomes more popular. The lower the barrier to entry the better, really. I know of some people who grant commit rights to anyone who lands a successful pull request.

Commenter jz made an interesting point that perhaps this is something that plagues web development for the most part. But as I argued, this is a problem that is endemic to all software.

The biggest open source project I am involved with is CouchDB, which is a document database written in Erlang. We have dependancies on specific versions of Erlang, various Erlang libraries, specific versions of OpenSSL, specific versions of SpiderMonkey, and so on. Which has caused us our fair share of problems. Even the C components are not safe from this sort of bitrot, with APIs that are modified or removed, and shared libraries that make backwards incompatible changes. My basic point was that the more dependencies you add to your project, the more you are going to have to work to keep up with that as a moving target. And it's worthwhile considering that for every dependency you add, you are also depending on every dependancy of that dependancy. These sorts of things can quickly increase exponentially.

I found an interesting post on what it's like to be a community manager by Jason Hibbets of Red Hat. And as I said in the comments, one of the things that we take for granted is that other people have a community mindset. But in fact, I think is our first problem. Problem zero, if you like.

I've been arguing strongly, recently, that healthy open source projects need people. You seem to be taking for granted that communities are worth having, but I think one of the big problems is that there are still so many people who aren't yet community minded.

You're looking, in other words, at the problem of being a community manager. But as a community manager, I also like to keep in mind what it feels like to be a developer who isn't part of a community. Because that's where the community comes from: people who weren't part of the community before. If you can think how they think, then you can boost your conversion rates and entice more people in.

Anyway. What do you think? Do you run your own open source project? Is it just you, or have you managed to attract contributors? How do you feel about sharing the maintenance tasks? And how aggressively do you attempt to attract contributors?

Noah Slater has been contributing to open source since 1999. He's an Apache member, O'Reilly author, and one of the folks behind CouchDB. He currently works for Engine Yard as a Community Engineer.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment