Last night, Brian Shirai unilaterally "ended" the RubySpec project, a sub-project of Rubinius (the alternative Ruby implementation which Brian was paid to work on full-time from 2007 to 2013). The blog post describing his reasons for "ending" the project led to a big discussion on Hacker News.
When a single, competing Ruby implementation tells that you its test suite is the One True Way, you should be skeptical. Charles Nutter, Ruby core committer and JRuby head honcho, spent a lot of time last night on Twitter talking to people about what this decision means. He's probably too busy and certainly too nice of a guy to write about what is a political issue in the Ruby community, so I'm going to do it on behalf of all the new or intermediate Rubyists out there that are confused by Brian's decision and what it means for us as a community.
Brian's Mission Is Good
A reference specification is a noble idea. Java has one, and it's probably a net positive for the language. If Ruby could magically have a complete implementation-agnostic language spec, today, I think everyone would be very happy with that outcome. I want to be clear that very few people in the Ruby community disagree with this idea in principle.
Practically speaking, all implementations of Ruby have absolutely benefited from RubySpec. JRuby runs RubySpec alongside the MRI suite. Many smaller implementations such as Topaz also use RubySpec. MRI uses RubySpec, most notably in the run-up to 1.9.2 (as Brian pointed out), but core members still use it today, even if it isn't officially part of the project (since all you need to do is run RubySpec against Ruby head).
Anyone that spends eight years of their life, for any reason, working in our community is someone we want to have around.
Brian's Technical Reasons Are Mostly Wrong
A lot of Brian's post is trying to spread fear, uncertainty, and doubt. Here's the statements of fact that I take issue with:
- Ruby has an ISO spec. Characterizing Ruby as a completely "spec-less" language would be incorrect. There's been an effort at an ISO spec for Ruby. Admittedly, it's not finished or perfect, and has mostly been the effort of a small group of Japanese academics. But it's there, and Matz has expressed that an ISO standard is what he sees as the way forward for a formal specification of Ruby.
- MRI's test suite has much more coverage now. It definitely used to be pretty bad, as Brian writes. The road from 1.8.7 to 1.9.3 was extremely, extremely painful for everyone, paved with bugs and minor versions. But to characterize the MRI suite as being inadequate for Ruby as a language seems inaccurate. JRuby runs almost 6000 tests and over 1 million assertions directly from the MRI suite. Does that sound like a test suite that is completely unusable to other Ruby implementations?
- MRI has used RubySpec, and some members still do. RubySpec's "minimum viable contribution" to Ruby has, and will continue to be, "does MRI pass RubySpec?". This doesn't take any official sanction or additional contribution from anyone - just run MRI against RubySpec, and open pull requests and issues when you find problems.
- A lot of MRI's process happens in Japanese. Matz, and a good chunk of the Ruby core team, is Japanese and runs on Japan time. While most of the discussion online happens in English, it can sometimes feel like there's "no process" around Ruby because a lot of discussion happens in Japanese or even in meatspace in Japan.
- Other languages use reference implementations. The absence of a formal specification, while not ideal, is not some dooming fate for a language. Python has got along just fine without one for quite a while.
- One bug doesn't change a lot. Brian pointed out that Ruby 2.2 fails a RubySpec and segfaults. This is obviously not intended - Ruby isn't supposed to segfault when running pure Ruby, ever. But one bug from a competing Ruby implementation (again, RubySpec is basically just Rubinius' test suite made re-usable) does not mean that MRI's suite is useless. All Ruby implementations have valid bug issues with MRI - and they open issues and PRs on MRI to deal with it. In fact, Brian refused to do this with this latest segfault (though he says he's opened tickets in the past). Charles Nutter had to open an bug report for him.
- Ugly tests are ugly tests, not useless ones. A lot of Brian's arguments come down to aesthetics - he thinks a lot of the MRI tests are nonsensical (some definitely are), the suite is incomplete (whose isn't?), it sets up test state inconsistently, combines too much behavior into a single test, and sometimes specify behavior that is undefined. All of these issues are valid, and they can all be fixed with a pull request. Brian's solution, instead, seems to be to throw the baby out with the bathwater and use (his) RubySpec instead.
Brian's Methods Are Controversial
I have to say that this is the tough part of this blog. Shit-slinging about personal slights and disagreements doesn't help move the issue forward, and I think the discussion about the ending of RubySpec has been very civil so far. However, there is a long history of non-technical reasons as to why Brian, Rubinius and RubySpec have been controversial in the Ruby community, and Ruby developers who haven't had time to follow the issue would be done a disservice if we all pretended that no one has personal issues with Brian Shirai.
- Brian doesn't value the contributions others have made to RubySpec. Brian doesn't think that the other committers to RubySpec have made meaningful contributions to the project. This attitude does not grow a community around an open-source project. It's actively hostile towards new contributors. It's not hard to imagine why RubySpec is struggling for contributions with a project lead that says things like:
- Brian wants more control over Ruby. For a long time, Brian has basically wanted a larger role in the decisionmaking and design process of Ruby, which, de facto, is Matz's sole privilege. This is where we get into the politics of RubySpec. Make no mistake, by making RubySpec the project (not "the test suite") "official" and requiring MRI developers to contribute to it, Brian is gaining control of the way Ruby is designed. I think it should be made clear that making RubySpec "official" in the way Brian wants is something he stands to gain from. Brian's issues with MRI's rejection of RubySpec are not technical, but political. This is OK (all large open source projects require some politicking to run), but Brian isn't making his aims and motivations clear in his blog post.
- Brian's primary responsibility is to Rubinius, not Ruby. Brian has said as much. In my opinion, he's also shown this in his actions in the past. Rubinius, his competing Ruby implementation, has shown that it's willing to break compatibility with MRI where it sees it is right to do so, or to "improve" the language in ways Brian and the Rubinius team thinks is right. This is, obviously, his perogative, but I think it's important to understand that Brian's vision of Ruby is obviously different than a lot of the core team, and Rubinius reflects this differing vision. Brian sees MRI as needing to get with his program, not the other way around.
You Can't Stop The Signal
In the end, RubySpec is open source software. It's sort of preposterous to end a project like this - Brian admits he made this decision unilaterally, and in the face of disagreement from his peers and contributors to RubySpec. Brian can't really "end" RubySpec so much as "leave the project". And it's a sad thing, I think - anyone that devotes this much of their lives to Ruby in any way is someone I wish would stay in the community. But Brian has clearly chosen to leave the mainstream of Ruby - and both MRI and Rubinius will suffer for it.
I think this whole episode carries a very poignant lesson about the way open source software is run. Brian seems to think that disagreements in OSS should be resolved with large, sweeping changes and formal governance - RubySpec and the proposed Ruby design process are concrete examples of this opinion. However, many successful projects, like Ruby and Rails, have operated with a much more piecemeal approach: incremental change, one pull request at a time.
RubySpec continues to exist, as code. Running RubySpec as a separate test suite, externally, and simply reporting the bugs back to MRI has always been and will continue to be useful. In fact, it seems like something good might come out of this episode after all - Charles Nutter (again, an MRI core committer) has discussed being open to including RubySpec in the MRI repo as a separate test suite, something which Brian has, to my knowledge, never proposed (I would guess since it would relinquish the control of the project to Matz and the Ruby core team).
Unfortunately, drive-by trolls and new Ruby developers will inevitably see this as "another sign Ruby is dying!!1". The reality is that Ruby has been doing quite well without RubySpec officially driving the design process. Ruby 2.2 is faster and more memory-efficient than ever, and a single failing RubySpec with a segfault doesn't change that.
So my advice to Ruby developers is this - let's put Ruby in Schrodinger's box. It's dead, it's alive, who cares? What I know is that Ruby is still a language that I love writing, and I still build awesome things with it. None of that has changed. And I don't think it will.
EDIT: Removed some unfair assertions about Rubinius' production-readiness, and made my point about the MRI suite's usefulness more clear.
Context: I actively work on Rubinius (for almost 2 years now), use it in
production for serious internet business. Take my views as you wish, but
consider I have at least some decent experience on the matters (instead of just
making them up).
From what I remember the ISO spec is largely based on Ruby 1.8 and does not
discuss critical features such as encodings. The ISO spec is also not freely
available (costing $200).
As a result of this the ISO specification is fairly useless when one wants to
support newer Ruby features such as encodings, keyword arguments, etc. Or
when one simply can't afford to buy it.
Having had my fair share of interacting with the MRI test suite, I can guarantee
you it's not "pretty good". One of many examples:
It took me and Charles over a month (if my memory serves me right) to implement
String#scrub for Rubinius/JRuby exactly due to this problem. The basic
documentation of the method was in the end more useful than the tests.
Is it better than before? Sure, but it's still a long way from the quality of
RubySpec or even the quality of specs you typically see in Gems.
People running RubySpec once in a while is completely useless when they don't
provide feedback on it, work on the specs itself, etc. This is like saying
you're contributing to the Linux kernel because you're just using it.
Python has the PEP system which helps tremendously. The argument of "X works for
others" does nothing to discredit any criticism on this. Just because somebody
else does something doesn't mean it works for others.
The bug is just a specific example. We've encountered plenty of cases in the
past that could've been dealt with a lot faster if everybody (MRI, JRuby, Rbx,
etc) were equally actively involved. Some examples:
They very much are useless for anybody who's not familiar with the intention,
either due to the lack of a proper description or something else.
In general I suggest you actually spend some time digging into any of the above,
in particular working with the MRI test suite. Especially before making the
Lets disect this for a moment to see what this refers to. We'll take a few of
the more common contributors in Ruby and see how much they've done for RubySpec
spec was added December 2nd this year, before that the last spec was in
commit on Jan 20, 2011
no commits yet in RubySpec itself, although he was working on several things
related to that, so props for that (I mean it)
the only JRuby developer that more actively works on RubySpec, although recent
changes were more maintenance related
maintenance related to get Opal running, which is appreciated although it
doesn't strictly help Ruby as a whole (as "mean" as that might sound)
Now lets look at some Rubinius developers:
should be obvious
mainly a bunch of CAPI stuff and smaller specs. I spend most time on the issue
tracker. A lot of the commits are from syncing Rbx <-> RubySpec.
If we limit Github's contribution graph to the last year we'll get the
The top 3 are all Rubinius developers, with eregon clocking in at #4. Meanwhile
the people that are most angry about RubySpec being pulled are at the very
bottom of the list.
From this it's rather clear that at least when it comes to working on RubySpec's
codebase, there's not a lot happening from non Rubinius developers. This is
exactly what Brian means with the lack of contributions, how changes matter (or
not) in the grand scale of things, etc.
He's also absolutely right about having to fix specs of others and he's not the
only one (I've done so in the past as well). People seem to get annoyed when we
demand quality, but it's the state of MRI's test suite that we actively want to
avoid, which can only be done by expecting people to actually write good specs.
This section is such bullshit I'm not even sure where you are getting this from.
The whole point was not for Brian, or any other individual for that matter, to
have more control over Ruby. Instead it's to ensure decisions are made
co-operatively, tested properly, discussed, etc, etc. These are all things that
are extremely important to ensure a good programming languages exists/continue
to exists. It's baffling that one expects businesses to bet millions of dollars
on a language that can't even get this right. Python for example at least has a
step in the right direction.
This is in reference to his attempts to work together (e.g. see the part about
getting RubySpec in the ISO). Given that MRI is outnumbered in terms of projects
using RubySpec (Topaz, Opal, JRuby, Maglev and Rubinius) this isn't really
No disagreement has been voiced. Most of those reviewing the article gave
feedback on the article itself but nobody disagreed that I know of. In fact,
I'm fairly sure most agree due to the complexity and energy drain of the
He can since he's the primary maintainer. People can of course fork the project
into something different, but then it's RubySpec-something, not RubySpec itself.
By this point I'm getting fed up with the amount of nonsense in this article,
but this is downright insulting. Brian has no interest in leaving Ruby, Rubinius
won't suddenly stop supporting Ruby where possible and Rubinius will anything
but suffer from it. If you are referring to Rubinius X, this is an idea to build
what we think would be "Ruby 10", but Rubinius itself will always continue to
Charles is but one of many in the MRI core team, his opinion is not the decisive
factor. Numerous attempts have been made in the past to ensure MRI adopts
RubySpec, with little success to date. It was rejected from the ISO spec, it's
not actively included in the development process and not a single contribution
has been made from any MRI developer (except for 1 commit of headius) in the
Bullshit, plenty of people have reported running Rubinius in production just
fine over the last few months. I myself run it in production and save for some
issues with third-party Gems it works perfectly fine. This argument of
"nontrivial deployments" goes to show how little you actually know on the
subject, making this article all the more confusing. In fact, one could argue
that Rubinius deployments are easier due to the support of C extensions instead
of requiring Java/FFI alternatives.
Open your eyes, businesses are moving away to languages for any number of
reasons. Twitter tried to make Ruby perform well with Kiji in the past, and they
moved away. Others are moving to languages with better concurrency facilities,
more stability, etc. If you think nothing will change it's time to open your
eyes and see the world as she truly is.