Last night, Brian Shirai unilaterally "ended" the RubySpec project, a sub-project of Rubinius (the alternative Ruby implementation which Brian was paid to work on full-time from 2007 to 2013). The blog post describing his reasons for "ending" the project led to a big discussion on Hacker News.
When a single, competing Ruby implementation tells that you its test suite is the One True Way, you should be skeptical. Charles Nutter, Ruby core committer and JRuby head honcho, spent a lot of time last night on Twitter talking to people about what this decision means. He's probably too busy and certainly too nice of a guy to write about what is a political issue in the Ruby community, so I'm going to do it on behalf of all the new or intermediate Rubyists out there that are confused by Brian's decision and what it means for us as a community.
A reference specification is a noble idea. Java has one, and it's probably a net positive for the language. If Ruby could magically have a complete implementation-agnostic language spec, today, I think everyone would be very happy with that outcome. I want to be clear that very few people in the Ruby community disagree with this idea in principle.
Practically speaking, all implementations of Ruby have absolutely benefited from RubySpec. JRuby runs RubySpec alongside the MRI suite. Many smaller implementations such as Topaz also use RubySpec. MRI uses RubySpec, most notably in the run-up to 1.9.2 (as Brian pointed out), but core members still use it today, even if it isn't officially part of the project (since all you need to do is run RubySpec against Ruby head).
Anyone that spends eight years of their life, for any reason, working in our community is someone we want to have around.
A lot of Brian's post is trying to spread fear, uncertainty, and doubt. Here's the statements of fact that I take issue with:
- Ruby has an ISO spec. Characterizing Ruby as a completely "spec-less" language would be incorrect. There's been an effort at an ISO spec for Ruby. Admittedly, it's not finished or perfect, and has mostly been the effort of a small group of Japanese academics. But it's there, and Matz has expressed that an ISO standard is what he sees as the way forward for a formal specification of Ruby.
- MRI's test suite has much more coverage now. It definitely used to be pretty bad, as Brian writes. The road from 1.8.7 to 1.9.3 was extremely, extremely painful for everyone, paved with bugs and minor versions. But to characterize the MRI suite as being inadequate for Ruby as a language seems inaccurate. JRuby runs almost 6000 tests and over 1 million assertions directly from the MRI suite. Does that sound like a test suite that is completely unusable to other Ruby implementations?
- MRI has used RubySpec, and some members still do. RubySpec's "minimum viable contribution" to Ruby has, and will continue to be, "does MRI pass RubySpec?". This doesn't take any official sanction or additional contribution from anyone - just run MRI against RubySpec, and open pull requests and issues when you find problems.
- A lot of MRI's process happens in Japanese. Matz, and a good chunk of the Ruby core team, is Japanese and runs on Japan time. While most of the discussion online happens in English, it can sometimes feel like there's "no process" around Ruby because a lot of discussion happens in Japanese or even in meatspace in Japan.
- Other languages use reference implementations. The absence of a formal specification, while not ideal, is not some dooming fate for a language. Python has got along just fine without one for quite a while.
- One bug doesn't change a lot. Brian pointed out that Ruby 2.2 fails a RubySpec and segfaults. This is obviously not intended - Ruby isn't supposed to segfault when running pure Ruby, ever. But one bug from a competing Ruby implementation (again, RubySpec is basically just Rubinius' test suite made re-usable) does not mean that MRI's suite is useless. All Ruby implementations have valid bug issues with MRI - and they open issues and PRs on MRI to deal with it. In fact, Brian refused to do this with this latest segfault (though he says he's opened tickets in the past). Charles Nutter had to open an bug report for him.
- Ugly tests are ugly tests, not useless ones. A lot of Brian's arguments come down to aesthetics - he thinks a lot of the MRI tests are nonsensical (some definitely are), the suite is incomplete (whose isn't?), it sets up test state inconsistently, combines too much behavior into a single test, and sometimes specify behavior that is undefined. All of these issues are valid, and they can all be fixed with a pull request. Brian's solution, instead, seems to be to throw the baby out with the bathwater and use (his) RubySpec instead.
I have to say that this is the tough part of this blog. Shit-slinging about personal slights and disagreements doesn't help move the issue forward, and I think the discussion about the ending of RubySpec has been very civil so far. However, there is a long history of non-technical reasons as to why Brian, Rubinius and RubySpec have been controversial in the Ruby community, and Ruby developers who haven't had time to follow the issue would be done a disservice if we all pretended that no one has personal issues with Brian Shirai.
- Brian doesn't value the contributions others have made to RubySpec. Brian doesn't think that the other committers to RubySpec have made meaningful contributions to the project. This attitude does not grow a community around an open-source project. It's actively hostile towards new contributors. It's not hard to imagine why RubySpec is struggling for contributions with a project lead that says things like:
- Brian wants more control over Ruby. For a long time, Brian has basically wanted a larger role in the decisionmaking and design process of Ruby, which, de facto, is Matz's sole privilege. This is where we get into the politics of RubySpec. Make no mistake, by making RubySpec the project (not "the test suite") "official" and requiring MRI developers to contribute to it, Brian is gaining control of the way Ruby is designed. I think it should be made clear that making RubySpec "official" in the way Brian wants is something he stands to gain from. Brian's issues with MRI's rejection of RubySpec are not technical, but political. This is OK (all large open source projects require some politicking to run), but Brian isn't making his aims and motivations clear in his blog post.
- Brian's primary responsibility is to Rubinius, not Ruby. Brian has said as much. In my opinion, he's also shown this in his actions in the past. Rubinius, his competing Ruby implementation, has shown that it's willing to break compatibility with MRI where it sees it is right to do so, or to "improve" the language in ways Brian and the Rubinius team thinks is right. This is, obviously, his perogative, but I think it's important to understand that Brian's vision of Ruby is obviously different than a lot of the core team, and Rubinius reflects this differing vision. Brian sees MRI as needing to get with his program, not the other way around.
In the end, RubySpec is open source software. It's sort of preposterous to end a project like this - Brian admits he made this decision unilaterally, and in the face of disagreement from his peers and contributors to RubySpec. Brian can't really "end" RubySpec so much as "leave the project". And it's a sad thing, I think - anyone that devotes this much of their lives to Ruby in any way is someone I wish would stay in the community. But Brian has clearly chosen to leave the mainstream of Ruby - and both MRI and Rubinius will suffer for it.
I think this whole episode carries a very poignant lesson about the way open source software is run. Brian seems to think that disagreements in OSS should be resolved with large, sweeping changes and formal governance - RubySpec and the proposed Ruby design process are concrete examples of this opinion. However, many successful projects, like Ruby and Rails, have operated with a much more piecemeal approach: incremental change, one pull request at a time.
RubySpec continues to exist, as code. Running RubySpec as a separate test suite, externally, and simply reporting the bugs back to MRI has always been and will continue to be useful. In fact, it seems like something good might come out of this episode after all - Charles Nutter (again, an MRI core committer) has discussed being open to including RubySpec in the MRI repo as a separate test suite, something which Brian has, to my knowledge, never proposed (I would guess since it would relinquish the control of the project to Matz and the Ruby core team).
Unfortunately, drive-by trolls and new Ruby developers will inevitably see this as "another sign Ruby is dying!!1". The reality is that Ruby has been doing quite well without RubySpec officially driving the design process. Ruby 2.2 is faster and more memory-efficient than ever, and a single failing RubySpec with a segfault doesn't change that.
So my advice to Ruby developers is this - let's put Ruby in Schrodinger's box. It's dead, it's alive, who cares? What I know is that Ruby is still a language that I love writing, and I still build awesome things with it. None of that has changed. And I don't think it will.
EDIT: Removed some unfair assertions about Rubinius' production-readiness, and made my point about the MRI suite's usefulness more clear.
I can see that I'm going to spend a good chunk of tomorrow (it's after 0130 local here now) going through Charles Nutter's Twitter discussions that Nate linked to. I've been paying attention to RubySpec for nearly five years now, seen the smoke clouds rising in the distance from the Ruby drama alluded to, and over time become both more convinced that we need something like RubySpec while at the same time increasingly questioning how much RubySpec can accomplish given the technical and other issues involved. I think I've had my attitudes adjusted somewhat, by Brian's action and attempt at explanation, and by Nate's very thoughtful piece here.
There's a lot here that I agree with, and not a little in Brian's post that I see as a wake-up call. I agree that It Would Be Very Nice if we had an ISO Standard; however, having seen the politics of that process up close and personal with more than one language, I can understand if Matz and the core team aren't in a rush to make a more comprehensive, post-1.8 ISO spec happen.
I agree with Brian where he suggests what is meant to be interpreted as the need for an implementation-neutral (or at least -independent), canonical set of specs for the language. They should evolve as the language continues to evolve, and can provide invaluable service while still approaching completeness, whatever that means. I too am sceptical about someone with his history with the community being given a, let alone the, leadership role in such an effort. That does not in any way diminish the community's, and industry's, imperative need for such an effort to be undertaken successfully.
We'd be in a much better place to defend ourselves against the keyboard trolls and professional haters that, as Nate said, will jump on this as though it's a gift unicorn showing up in their bedrooms, or at least their news feeds, with a comprehensive, independent set of specs. That's not the best reason to make such specs happen, of course. Our team, like a justifiably huge part of the community, uses MRI (though we're staying with 2.1.5 for now). But the scope and complexity of Ruby 2.2, compared with versions only a couple of years out of date, seem to argue for community organisational change.
Proposal: Keep MRI as the reference implementation of Ruby, and absolutely keep Matz as BDL. But can we all agree that a reference implementation shouldn't be a one-off, but should show others how it's done right? As new features are introduced into MRI Ruby, ask Matz's designated release maintainer to verify that reference specs exist for those new features, in a way that isolates (necessary) implementation dependencies (like the C standard library mentioned by both Brian and Nate). Let's have a consortium, or foundation-type org, take over maintainership of RubySpec or a fork thereof, and steer it towards something that everybody can agree is a Good Thing that isn't threatening because they have as much of a stake in it as anyone else. Get some "community guidelines" with teeth courtesy of "you engage in 'slanderous or duplicitous behaviour', you lose the right to consortium-held IP separate from RubySpec2-the-open-source-system (like trademarks, branding)." Pull a Mozilla, and dare the would-be Brians to push an Iceweaselised "Ruby" onto users who can at least see the difference and, with a bit of searching, understand why it is.
Maybe we'd evolve to something that would make an ISO Standard almost anticlimactic. Maybe the community would eventually decide that they want to hang onto more control over Ruby's destiny than an ISO Standard would give them. Those are battles for another day, if not another decade. For now, there are really only two questions:
What is "Ruby"? And can we prove it in an objective rather than de facto manner?
EDIT: Tpyos.