Last night, Brian Shirai unilaterally "ended" the RubySpec project, a sub-project of Rubinius (the alternative Ruby implementation which Brian was paid to work on full-time from 2007 to 2013). The blog post describing his reasons for "ending" the project led to a big discussion on Hacker News.
When a single, competing Ruby implementation tells that you its test suite is the One True Way, you should be skeptical. Charles Nutter, Ruby core committer and JRuby head honcho, spent a lot of time last night on Twitter talking to people about what this decision means. He's probably too busy and certainly too nice of a guy to write about what is a political issue in the Ruby community, so I'm going to do it on behalf of all the new or intermediate Rubyists out there that are confused by Brian's decision and what it means for us as a community.
A reference specification is a noble idea. Java has one, and it's probably a net positive for the language. If Ruby could magically have a complete implementation-agnostic language spec, today, I think everyone would be very happy with that outcome. I want to be clear that very few people in the Ruby community disagree with this idea in principle.
Practically speaking, all implementations of Ruby have absolutely benefited from RubySpec. JRuby runs RubySpec alongside the MRI suite. Many smaller implementations such as Topaz also use RubySpec. MRI uses RubySpec, most notably in the run-up to 1.9.2 (as Brian pointed out), but core members still use it today, even if it isn't officially part of the project (since all you need to do is run RubySpec against Ruby head).
Anyone that spends eight years of their life, for any reason, working in our community is someone we want to have around.
A lot of Brian's post is trying to spread fear, uncertainty, and doubt. Here's the statements of fact that I take issue with:
- Ruby has an ISO spec. Characterizing Ruby as a completely "spec-less" language would be incorrect. There's been an effort at an ISO spec for Ruby. Admittedly, it's not finished or perfect, and has mostly been the effort of a small group of Japanese academics. But it's there, and Matz has expressed that an ISO standard is what he sees as the way forward for a formal specification of Ruby.
- MRI's test suite has much more coverage now. It definitely used to be pretty bad, as Brian writes. The road from 1.8.7 to 1.9.3 was extremely, extremely painful for everyone, paved with bugs and minor versions. But to characterize the MRI suite as being inadequate for Ruby as a language seems inaccurate. JRuby runs almost 6000 tests and over 1 million assertions directly from the MRI suite. Does that sound like a test suite that is completely unusable to other Ruby implementations?
- MRI has used RubySpec, and some members still do. RubySpec's "minimum viable contribution" to Ruby has, and will continue to be, "does MRI pass RubySpec?". This doesn't take any official sanction or additional contribution from anyone - just run MRI against RubySpec, and open pull requests and issues when you find problems.
- A lot of MRI's process happens in Japanese. Matz, and a good chunk of the Ruby core team, is Japanese and runs on Japan time. While most of the discussion online happens in English, it can sometimes feel like there's "no process" around Ruby because a lot of discussion happens in Japanese or even in meatspace in Japan.
- Other languages use reference implementations. The absence of a formal specification, while not ideal, is not some dooming fate for a language. Python has got along just fine without one for quite a while.
- One bug doesn't change a lot. Brian pointed out that Ruby 2.2 fails a RubySpec and segfaults. This is obviously not intended - Ruby isn't supposed to segfault when running pure Ruby, ever. But one bug from a competing Ruby implementation (again, RubySpec is basically just Rubinius' test suite made re-usable) does not mean that MRI's suite is useless. All Ruby implementations have valid bug issues with MRI - and they open issues and PRs on MRI to deal with it. In fact, Brian refused to do this with this latest segfault (though he says he's opened tickets in the past). Charles Nutter had to open an bug report for him.
- Ugly tests are ugly tests, not useless ones. A lot of Brian's arguments come down to aesthetics - he thinks a lot of the MRI tests are nonsensical (some definitely are), the suite is incomplete (whose isn't?), it sets up test state inconsistently, combines too much behavior into a single test, and sometimes specify behavior that is undefined. All of these issues are valid, and they can all be fixed with a pull request. Brian's solution, instead, seems to be to throw the baby out with the bathwater and use (his) RubySpec instead.
I have to say that this is the tough part of this blog. Shit-slinging about personal slights and disagreements doesn't help move the issue forward, and I think the discussion about the ending of RubySpec has been very civil so far. However, there is a long history of non-technical reasons as to why Brian, Rubinius and RubySpec have been controversial in the Ruby community, and Ruby developers who haven't had time to follow the issue would be done a disservice if we all pretended that no one has personal issues with Brian Shirai.
- Brian doesn't value the contributions others have made to RubySpec. Brian doesn't think that the other committers to RubySpec have made meaningful contributions to the project. This attitude does not grow a community around an open-source project. It's actively hostile towards new contributors. It's not hard to imagine why RubySpec is struggling for contributions with a project lead that says things like:
- Brian wants more control over Ruby. For a long time, Brian has basically wanted a larger role in the decisionmaking and design process of Ruby, which, de facto, is Matz's sole privilege. This is where we get into the politics of RubySpec. Make no mistake, by making RubySpec the project (not "the test suite") "official" and requiring MRI developers to contribute to it, Brian is gaining control of the way Ruby is designed. I think it should be made clear that making RubySpec "official" in the way Brian wants is something he stands to gain from. Brian's issues with MRI's rejection of RubySpec are not technical, but political. This is OK (all large open source projects require some politicking to run), but Brian isn't making his aims and motivations clear in his blog post.
- Brian's primary responsibility is to Rubinius, not Ruby. Brian has said as much. In my opinion, he's also shown this in his actions in the past. Rubinius, his competing Ruby implementation, has shown that it's willing to break compatibility with MRI where it sees it is right to do so, or to "improve" the language in ways Brian and the Rubinius team thinks is right. This is, obviously, his perogative, but I think it's important to understand that Brian's vision of Ruby is obviously different than a lot of the core team, and Rubinius reflects this differing vision. Brian sees MRI as needing to get with his program, not the other way around.
In the end, RubySpec is open source software. It's sort of preposterous to end a project like this - Brian admits he made this decision unilaterally, and in the face of disagreement from his peers and contributors to RubySpec. Brian can't really "end" RubySpec so much as "leave the project". And it's a sad thing, I think - anyone that devotes this much of their lives to Ruby in any way is someone I wish would stay in the community. But Brian has clearly chosen to leave the mainstream of Ruby - and both MRI and Rubinius will suffer for it.
I think this whole episode carries a very poignant lesson about the way open source software is run. Brian seems to think that disagreements in OSS should be resolved with large, sweeping changes and formal governance - RubySpec and the proposed Ruby design process are concrete examples of this opinion. However, many successful projects, like Ruby and Rails, have operated with a much more piecemeal approach: incremental change, one pull request at a time.
RubySpec continues to exist, as code. Running RubySpec as a separate test suite, externally, and simply reporting the bugs back to MRI has always been and will continue to be useful. In fact, it seems like something good might come out of this episode after all - Charles Nutter (again, an MRI core committer) has discussed being open to including RubySpec in the MRI repo as a separate test suite, something which Brian has, to my knowledge, never proposed (I would guess since it would relinquish the control of the project to Matz and the Ruby core team).
Unfortunately, drive-by trolls and new Ruby developers will inevitably see this as "another sign Ruby is dying!!1". The reality is that Ruby has been doing quite well without RubySpec officially driving the design process. Ruby 2.2 is faster and more memory-efficient than ever, and a single failing RubySpec with a segfault doesn't change that.
So my advice to Ruby developers is this - let's put Ruby in Schrodinger's box. It's dead, it's alive, who cares? What I know is that Ruby is still a language that I love writing, and I still build awesome things with it. None of that has changed. And I don't think it will.
EDIT: Removed some unfair assertions about Rubinius' production-readiness, and made my point about the MRI suite's usefulness more clear.