Create a gist now

Instantly share code, notes, and snippets.

Turning Off Github Issues

My friend Michael Jackson turned off github issues on one of his smaller projects. It got me thinking...

Maintainers getting burned out is a problem. Not just for the users of a project but the mental health of the maintainer. It's a big deal for both parties. Consumers want great tools, maintainers want to create them, but maintainers don't want to be L1 tech support, that's why they burn out and everybody suffers.

The solution is to get more people involved, but how? Some of us think that letting people open issues helps. I don't. Most issues opened are bogus: an incorrect module import, not knowing that promises swallow errors and incorrectly assuming it's the project's problem, or they're questions that are already answered in the README or docs.

This isn't a rigorous statistic, but I'm pretty sure these are 99% of the issues on Github.

Issues aren't creating contributors, and they aren't really helping the project's code either. You're either spending time closing bogus issues or you're trying to find other people to do it, either way it's not healthy for any of you.

What if we just turned off the issues tab?

Consider this: issues are for powerless customers. You open an issue with your software vendor because you don't have the power to fix the problem. But programmers, using open source, do.

Pull Requests for Everything

Opening a Github issues doesn't require you to learn anything new, write any code, or even see the source code of the project. If we turn off issues, then the only feedback mechanism is a pull request. Let's explore the only use-cases I can think of for issues, and how they can be handled with pull requests.

Think you found a bug?

Fork the repo, add a failing test. This will encourage devs to learn how to write tests, and in the process, maybe discover that the test passes and something is broken in their app.

Too early in your career to take on writing a test case? Add an example in the repo that illustrates the bug. If you are able to write enough code to think you've found a bug and open an issue, you are able to open the examples/ directory of a project and make a new one that shows the bug. You may, again, find that the bug is in your app, saving the mental health of the maintainers from wild goose chases for bugs that don't exist. Or, you'll make the pull request, the maintainer can run your example and see "ah there is a bug" and then fix it.

Using issues for this doesn't give the developer any new experience and usually leads to the maintainers chasing down bugs that don't exist.

Using a pull requests with a test case or an example gives the developer new experience, puts the project's code into their editor, and validates or invalidates their bug before ever asking for the free time of the maintainers.

If we want to include more people in open source, I can't think of a better way than getting open source code on their machines and into their editors.

Want to discuss a feature?

Talking about code is often fruitless until there's some actual code to talk about. Pull requests require some code in order to start a conversation.

Write a proof of concept and make a pull request. Nothing is better than code that kind of works when discussing features. It reveals shortcomings of the current code and helps everybody understand better what it's going take.

Don't have time or know-how to write a PoC? write some tests that illustrate the feature. You don't need an implementation, just write some assertions that show the API you think would be good. Or, write an example that doesn't work. These options let everybody involved know exactly what you're trying to do and the conversation will go way faster.

It's social coding. Let's have our conversations start with code, working or otherwise. Let's build up contributors by getting some OSS in their editor, and get them writing some code.

Have a question?

Okay, we can't do pull requests for this. But issues are a terrible place for questions anyway. Read the docs, read the examples, go to stack overflow. Questions aren't issues.

Whatever community of folks answering questions you want to build you can build on a platform designed for it. Maintainers can look at the questions over there to get a feel for where their documentation is lacking, but you don't need the issue tracker cluttered up overwhelming maintainers. Turning off issues forces this optimal scenario.

@kentcdodds

I'm starting to buy in. Still trying to let this sink in and think of the use cases that I've had for issues that this doesn't quite cover. Discussions about project roadmap or other non-code related aspects of a project need to happen somewhere.

@ryanflorence
Owner

pull request to ROADMAP.md

@ryanflorence
Owner

"non-code related aspects of a project" what are those?

@tylermcginnis

Fork the repo, add a failing test. This will encourage devs to learn how to write tests, and in the process, maybe discover that the test passes and something is broken in their app.

I'm 100% with this if, by switching to this model, the repo also puts emphasis and instruction on how to actually "add a failing test". If that's done well enough for a beginner to participate, it's a win/win on all counts.

@kentcdodds

Here's an example of a useful issue and I'm worried that the issue wouldn't have been reported if they'd been required to file an issue. They would have just moved on to another project and I would be short a contributor. Thoughts?

@ryanflorence
Owner

^ yeah, the project will naturally get easier to contribute to I think if I've got a mindset that people are going to be looking at my tests more often.

@kentcdodds

One thing that I think a project should definitely have is a big section in the README.md that says something like:

# Questions & Support

If you have any questions or need help, please go to [StackOverflow](http://stackoverflow.com/) or [chat](http://chat.example.com). This project does not utilize issues. If you feel like you've found a bug or would like to request a feature, please see [`CONTRIBUTING.md`](CONTRIBUTING.md).

And in the CONTRIBUTING.md, it needs to take into account that some people are new to programming/github and provide resources to help them get ramped up with git, GitHub, etc. For example, screencasts for how to set up the project are extremely helpful. And you could reference makeapullrequest.com :-)

@knowbody

I totally agree with this model

@ryanflorence
Owner

If you've got a repo that has npm run validate as API, you should have a test around it.

https://github.com/ryanflorence/react-project/blob/master/tests.js#L47

If that guy found a potential bug, he could add a pending test:

describe('npm run validate', () => it('runs without an error'))

Hopefully your project has tests similar to it so he might even be able to fill it in and show that it fails.

@kentcdodds

More context:

In my CONTRIBUTING.md I have project setup steps for new contributors:

## Steps

1. File an issue and discuss your proposed solution first
2. Fork and clone the repo
3. `$ npm install` to install dependencies
4. `$ npm run validate` to validate you've got it working
... etc.

There aren't really any tests around npm scripts. It's not part of the API. This is to help onboard new contributors. So I'm thinking that it'd be good to just tell people if that doesn't work, just keep going and submit the PR anyway? That'd probably work... Other thoughts?

@aaronfrost

I think that this is a great idea! Very well stated as well.

@kentcdodds

Oh, and where do you list TODOs? In TODO.md? Seems like it'd be difficult to coordinate who's working on what... Hmmm... Maybe you could make it a table with columns for who's working on it or something? Seems like that part would just be a bit too much work.... Still mulling this over.

@eanplatter

I was pretty against this at first, but it's actually pretty logical. A feature discussion around a PR is a lot more productive than an issue. Same with bugs.

Also with beginners, you may have less participation, but their participation is more effective. Issues can almost give a beginner a false sense of productivity. You'll learn a lot more making a PR than you would making an issue.

It pushes the investment of time onto to one with the issue rather than the maintainer, I can't count the hours I spend trying to recreate bugs in my projects' issues.

@kentcdodds

I was pretty against this at first

Welcome to the club. :-)

Issues can almost give a beginner a false sense of productivity.

I can see how that could be true. Though getting legitimate bug reports is more helpful than nothing at all. But even worse is the maintainers getting burnt out.

@jlongster

Oh, and where do you list TODOs? In TODO.md? Seems like it'd be difficult to coordinate who's working on what... Hmmm... Maybe you could make it a table with columns for who's working on it or something? Seems like that part would just be a bit too much work.... Still mulling this over.

Github Issues is known to be a poor at actual project management. I think a roadmap or a todo document makes sense for small projects, and if you need anything more you should use software more focused on project management. I know Babel uses phabricator.

@kentcdodds

Ah, true, I'll bet that a trello board would actually work pretty well for this scenario. Something that's publicly visible but only editable by people who have committed code to the project πŸ‘

@eanplatter

:D

I was thinking about adding a bug-recreation example that is almost a sandbox for showing bugs. Because half the time the bugs are on the users end, probably more than half.

You'd just have to be clear in your docs on how to submit a bug report. Maybe a bug template?

@mjackson

I was pretty against this at first

Welcome to the club. :-)

IIRC Ryan initially scoffed at the idea when I told him about it yesterday... πŸ˜… It takes some getting used to, and I think the way Ryan explains his thinking here helps to make that transition. Good stuff.

@eanplatter

We use Zenhub which gives you a Kanban type of board in github with a chrome extension. It's been pretty good, and is free for OSS.

@eanplatter

IIRC Ryan initially scoffed at the idea when I told him about it yesterday... πŸ˜…

It's just one of those ideas, like putting markup in your JavaScript πŸ˜› Sounds bad at first, ends up not being so crazy.

@kentcdodds

ends up not being so crazy.

Note, I don't know of any significantly popular project actually trying this out and having success, so I think it's a little early to say this isn't crazy ;-)

@eanplatter

Note, I don't know of any significantly popular project actually trying this out and having success

Did Babel remove issues? Or did they just end up moving them to a different service?

@kentcdodds

They moved to phabricator. Which has different features, but is pretty much the same idea I think.

@Nargonath

I'm pretty new to open-source and I try to contribute to projects as much as possible. I'm not a maintainer of a popular OSS project but I totally understand the burden that might come with it. I wonder if closing the issues tab would not lower the interest people address towards that particular project. Some project come with a big learning curve and personally I have already spent a few evenings trying to understand a project code base and how it works just to be able to contribute on an issue. I'm not sure everyone who used to submit issues related to a feature request or a bug will be willing to take the time needed to submit a PR. Not every developer develops as a hobby and some just do it on their work time. It may be ok to take the time to submit an issue during your working hours but I don't think they will take the time for a PR (unless they decide to do it on their spare time). I am not saying that closing the issue tab is a bad idea but just that I am mitigated about the consequences.

Perhaps a compromise would be to have a separate tab used to address "Questions" so we could keep the questions issues related out of the issue zone but that requires some work from Github and who knows when it would happen. Another option would be to use Gitter. I used to post issues when I had questions I couldn't find answer to as I saw many people doing it so I thought it was an ok thing to do. But now I just pop to Gitter and ask around instead. I know that these options will not lower the amount of "useless" interactions with the project and could probably just move the problem somewhere else. This is just my 2 cents.

@Nargonath

After reading the answers that arises while I was writing my previous post, it does make sense to remove the issue tab.

Also with beginners, you may have less participation, but their participation is more effective. Issues can almost give a beginner a false sense of productivity. You'll learn a lot more making a PR than you would making an issue.

I agree on this but I wonder what would be the conversion rate.

@kentcdodds

I'm not sure everyone who used to submit issues related to a feature request or a bug will be willing to take the time needed to submit a PR. Not every developer develops as a hobby and some just do it on their work time.

I think you'll find many people will have a similar feeling about this comment: if they aren't willing to take the time to submit a PR, then they must not need it enough. OSS contributors are not around to do your work for you :-) If you need it done for your job, then you do it.

I think a prerequisite to doing something like this is making your repo contribution instructions as helpful and beginner-friendly as possible. But honestly, it's OSS, and most of us are doing this on our free time :-) But if you want contributions, then people need helpful instructions.

@Nargonath

I think you'll find many people will have a similar feeling about this comment: if they aren't willing to take the time to submit a PR, then they must not need it enough. OSS contributors are not around to do your work for you :-) If you need it done for your job, then you do it.

You are definitely right. I hope my first comment didn't feel aggressive or something alike. As I'm not a native English speaker sometimes my sentences may sound aggressive whereas it is far from my desire. :-/

@kentcdodds

As I'm not a native English speaker sometimes my sentences may sound aggressive whereas it is far from my desire. :-/

No worries! Dealing with a global industry we all have to give one another the benefit of the doubt due to culture/language barriers/etc :-)

@awolf
awolf commented Mar 17, 2016

@kentcdodds without issues you are going to need the todo.md. Especially for new contributors and aligning their efforts. I would have never worked on the JavaScriptAir without a way to collaborate with you and other contributors.

@substack

I find as a consumer of open source modules, if I can't find an easy way to submit a bug report or see what bugs are known, I start looking for alternatives. As a maintainer, I don't worry much about issues piling up or getting spammed with nonsense. Turn off email notifications and it's not a problem. I think it's better that people have an outlet to coordinate without the substantial hurdle of code upfront than to not have a way for people to discuss their problems at all.

@mattecapu

Perhaps a compromise would be to have a separate tab used to address "Questions"

+1 to that.
I try to be as "PR friendly" as possible when reporting bugs/asking changes, but sometimes you just don't know how to tackle a particular problem, no need to change anything.

Moreover, sometimes spending time writing a proof-of-concept for a feature that it's immediately dismissed by the mantainers can be frustrating... but I guess PRs to ROADMAP.md easily fix this.

@kentcdodds

@awolf

I would have never worked on the JavaScriptAir without a way to collaborate with you and other contributors

Remember, that collaboration took place on a PR ;-) (and other collaborations could have)

Though I think that the JavaScript Air website repo may not fall under this same category of OSS libs...

@Nargonath

@kentcdodds Thanks. I was just trying to take the point of view of someone not ready to invest the time needed but that was not my personal stand. If we all go in the same way there would be no debate. :)

@mackenza

if GH issues are causing you burnout... turn them off. Just as the way you run an open source project is a matter of your own personality, free will, and processes, so should be how you would like to get feedback from the community. So if they are 99% bogus, causing you burnout, feel free to turn them off.

But that doesn't mean that the next project maintainer is inviting burnout because she leaves GH Issues on. Nor does turning off GH Issues require any consensus as being a good idea or a bad idea among project maintainers. It can simply be the way that you prefer to deal with your community and the way you prevent burnout.

To each his own.

@thedamon

Questions could also be addressed by submitting a PR to the documentation with an attempt at explaining how to do something that gets turned down by the maintainer with a description of how to actually do it. (or comments on commits to the docs along the lines of -- "could we add a description of how to..."??)

Although this could get pretty passive aggressive ("There is no easy way to do X"), it would make docs more like a community wiki that is constantly changing and more likely to cover common use-cases, gotchas, etc

@Pajn
Pajn commented Mar 17, 2016

Sometimes I find issues or feature requests that I would like to get fixed sometime but they aren't pressing so I might not have time to fix them right away. kentcdodds/validate-commit-msg#25 is an example of one of those, while I plan to get to it sometime I haven't had time yet, and until then I like to have it noted somewhere.

However I totally understand that one or a few people can't provide free support to everyone, but "bad" issues could just be blindly closed or ignored.

Sure, GH issues aren't the perfect place for memos either and could easily be placed elsewhere as everyone doesn't need to see them, but they are nicely packed together with the correct repo :)

@jimthedev

For what it is worth in terms of historical precedence Laravel (a large popular PHP framework) did this. There was a lot of Twitter and Reddit chatter on this and I think they've since brought them back. Worth investigating what worked and what didn't.

https://m.reddit.com/r/PHP/comments/2fx0sz/laravel_removing_github_issues_opting_for_failing/

@fredguest

it looks like Laravel stuck with the decision to turn issues off https://github.com/laravel/laravel

@hzoo
hzoo commented Mar 17, 2016

Very interesting! I guess we are already doing this in Babel, like what @amasad mentioned on twitter.

Asking for a test repo is can be great for maintainers (instead of just the input/output code on an issue). Hopefully users will be encouraged to do so and aren't turned off by doing so. At least for Babel, making a test case in code isn't that much harder since it's just an actual/expected file of code. I've added a testing label https://github.com/babel/babel/pulls?q=is%3Aopen+is%3Apr+label%3Atesting, and started up a simple test writing section in CONTRIBUTING, but now I think we should one in more depth (with pictures?).

Making a ROADMAP.md might be cool - making it as an issue might not be the best https://phabricator.babeljs.io/T7154

@tuxtina
tuxtina commented Mar 17, 2016

Purely from a "user" perspective, I have found GitHub issues extremely helpful. It is possible to get a quick overview of open bugs, including those issues that don't have a reproducible test case yet. And when starting out using a project, just reading through open and closed issues often helped me with the inevitable "intended behavior" vs. "bug" vs. "using it wrong" uncertainties.

With the one project I encountered in the last few months that had issues turned off, I was unable to get the same level of info from digging through the project's pull requests, and there were next to no discussions on StackOverflow (though I don't think the maintainer points people to StackOverflow).

I do not maintain a large project on GitHub, so I am entirely unqualified to comment on the effects that the presence of the issues tab has on maintainer sanity ;) I just know that I would miss it.

You open an issue with your software vendor because you don't have the power to fix the problem. But programmers, using open source, do.

I certainly do not feel confident that I can understand the inner workings of every open source module I use, or make modifications without breaking everything. Or in some cases, even getting it to build on my machine ;)

@thelambdaparty

This is fantastic, thanks for the post! _Fork the repo, add a failing test_. The idea takes "test driven development" to a new level πŸ˜„

For me, removing issues from a repo is like removing comments _ from your blog. It might sound crazy or negative at first but it actually may increase the quality of your posts, as the comments are left to other platforms (twitter, reddit, hn) and it encourages others to write another blog post as reply, just like one would do a pull request instead of an issue.

Because the investment time to do a blog post or pull request is higher there is an expected higher quality from it, as it's easier to go to stackoverflow than to do a pull request just for "this doesn't work in my config"

This is why i love the community, these ideas are great 😁

@gre
gre commented Mar 18, 2016

I agree with @substack's points, but I'm starting to buy your approach too, I like how it requires people to get a bit more involved in the project (via PR).
In github issues, you usually see bunch of them that are not "real" bug reports, either because it's a question or because it's a "maybe there is a bug" and involves you to look into it.
I think the great thing about your approach is that it redirects (by design) all time consuming things to the community (question will go into stackoverflow, chat can be used to investigate on a bug, and test case showing the case will be very helpful and productive rather than these "maybe there is a bug" ones).

It probably works great for "core" libraries but maybe less for projects.
It's very important that README clearly define and explain things (kinda like in https://github.com/babel/babel one) because users will go crazy about it not understanding the approach (and it's a big risk for this approach, to have the project losing popularity).

@thelambdaparty

I was having a look at a TypeScript issue where they discuss about a feature, and thought how that would work without issues.

By what @ryanflorence said, a pull request can be made to suggest a feature. I thought about the following: Having a repo to discuss project features (or in the project's repo depending on the size) with the following content:

- Examples/       // code examples of suggestions
- ROADMAP.md      // features agreed on
- SUGGESTIONS.md  // suggested features that are in discussion
- README.md       // explaining the repo, links to chat room, e.t.c

Then discussions can happen in Gitter/Slack/IRC, and notes/links be added to the files so others can catch-up or review past discussions (sort of like TC39 notes)

@lukeschunk

I agree that - to a beginner - writing a test can seem super daunting. If you don't want beginners to contribute - that's understandable I suppose, but even better in my opinion would be to do what Tyler said and write documentation on a recommended way to write a test.

@revolunet

Most issues are trash on popular projects but its a goldmine of informations and a place for community exchange that hopefully leads to PRs at some point; We can make them more useful by providing templates and enforcing rules (ex: submit bug = submit test, jsbin or whatever reproduction).

Github should have a stackoverflow integration at some point for "questions" though.

Github issue checker bots could help reduce the noise too.

@spudly
spudly commented Mar 19, 2016

I like the recommended approach for users to create a pr with failing tests, but I think removing the issues tab is too extreme.

For me, even moving issues to a different place (like Babel did) is a barrier to contribution. I don't like having to create a whole new user account with a different website just to report a bug. I've run into multiple bugs in Babel that I have not reported because it's easier to work around the issue than it is to report it.

I say if it's too much work to keep up with the issues, you should just say so in your readme and state that if they actually want something worked on they should submit a PR. At least then the users could still keep track of known issues (and workarounds!), even if you don't have time to fix them.

@thelambdaparty

Issues might still be useful for bug tracking, like @spudly said. I though about a possible workflow:

found a bug ->
- pull request with failing test (using semantic-release)
- Travis build fails, automatically create issue with details
- Use issue to track/discuss the bug

Then the post build process that creates the issue can have a defined format (how to structure the issue description). Any issue which was not created by the build process (someone manually created it) is automatically closed, with a reply explaining about how to contribute, e.t.c.

This came up because i remembered of people who are beginning in open-source trying to find issues which are labeled easy/beginner/quick e.t.c, so it's good for them and for bug tracking.

@halt-hammerzeit

Ha ha, closing issues would be the worst that could happen with a project.
"Closing issues" = "Project death"

Don't want to answer to issues yourself?
Give others the privileges.

By closing the issues you are performing FASCHISM.
It's a free society!
It's a free country!
No more totalitarian project owners!
THE CODE MUST BE FREE
OPEN SOURCE MUST BE LIBERATED

@elgubenis

I think creating Issues and getting contributors are 2 different things, because 1 is creating Issues and the other is getting contributors, wow ... mind === mindStates.blown

@taion
taion commented Mar 30, 2016

Got linked here via @knowbody's new repo.

My experience with smaller libraries actually is the opposite here. I'm very happy to get issues (even useless ones) on small repos. They're a net value-add to me, even when they just create more work and don't come with fixes.

Why? Because I find a lot of value in discussing use cases with users. I look at questions on the code and usage as being opportunities to refine immature packages. Often I'm not sure that I'm doing the right thing, and even without code contributions, just questions, really what we might otherwise dismiss as "useless" issues are very useful because they point me toward rough spots in my API.

Also, if the package is small and immature, I don't necessarily want my users to submit unsolicited PRs. Maybe there are things in my roadmap that I haven't written out, or considerations that I haven't fully shared yet, whatever – I don't want to force people to write out a fully coded solution, when especially for smaller libraries, discussion is actively valuable. This is like the Stack Overflow XY problem – in talking through things with the user, I can find better general principles, that maybe are cleaner or accomplish more than a strict "fix".

Moreover, you're not going to get away from being L1 support for libraries where you're the sole maintainer anyway.

For large, mature repos, these things don't hold, and I think the problems with GitHub issues are more visible, but I've gotten a lot of use out of GitHub issues on smaller repos.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment