Christmas day, 2015. I woke up in the south of Ukraine - in the very same apartment where I'd lived for a month back in the spring, hacking on the NFG representation of Unicode. NFG was just one of the missing pieces that had to fall into place during 2015 in order for that Christmas - finally - to bring the first official release of the language we now know as Raku.
I sipped a coffee and looked out onto a snowy courtyard. That, at least, was
reassuring. Snow around Christmas was relatively common in my childhood. It
ceased to be the year after I bought a sledge. I opened my laptop, and took
a look at the #perl6-dev IRC channel. Release time would be soon - and I
would largely be a spectator.
My contributions to the Rakudo compiler had started eight years prior. I had no idea what I was getting myself into, although if I had known, I'm pretty sure I'd still have done it. The technical challenges were, of course, fascinating for somebody who had developed a keen interesting in languages, compilers, and runtimes while at university. Larry designs languages with an eye on what's possible, not on what's easy, for the implementer. I learned, and continue to learn, a tremendous amount by virtue of working on Raku implementation. Aside from that, the regular speaking at workshops and conferences opened the door to spending some years as a teacher of software development and architecture, and after that left me with a wealth of knowledge to draw on as I moved on to focus on consultancy on developer tooling. Most precious of all, however, are the friendships forged with some of those sharing the Raku journey - which I can only imagine lasting a lifetime.
Eight years had gone by surprisingly quickly. When one is involved with the day-to-day development, the progress is palpable. This feature got done, this bug got fixed, this design decision got made, this library got written, this design decision got re-made for the third time based on seeing ways early adopters stubbed their toe, this feature got re-implemented for the fourth time because of the design change... From the outside, it all looks rather different; it's simply taking forever, things keeping getting re-done, and the developers of it "have failed us all" (all seasons have their grinches).
Similarly, while from the outside the Christmas release was "the big moment", from the inside, it was almost routine. We shipped a Rakudo compiler release, just as we'd been doing every month for years on end. Only this time, we also declared the specification test suite - which constitutes the official specification of the language - as being an official language release. The next month would look much like the previous ones: more bug reports arrive, more things get fixed and improved, more new users show up asking for guidance.
The Christmas release was the end of the beginning. But the beginning is only the start of a story.
It's been five years since That Christmas. Time continues to fly by. Each week brings its advances - almost always documented in what's currently known as the Rakudo Weekly, the successor of the Perl 6 Weekly. Some things seem constant: there's always some new bug reports, there will always be something that's under-documented, no matter what you make fast there will always be something else that is still slow, no matter how unlikely it seemed someone would depend on an implementation detail they will have anyway, new Unicode versions require at least some effort, and the latest release of MacOS seemingly always requires some kind of tweak. Welcome to the life of those working on a language implementation that's getting some amount of real-world use.
Among the seemingly unending set of things to improve, it's easy to lose sight of just how far the Raku Programming Language, its implementation in Rakudo, and the surrounding ecosystem has come over the last five years. Here I'll touch on just a few areas worthy of mention.
Maturity of a language implementation, its tools, and its libraries, is really, really, hard won. There's not all that many shortcuts. Experience helps, and whenever a problem can be avoided in the first place, by having somebody about with the background to know to do so, that's great. Otherwise, it's largely a case of making sure that when there are problems, they get fixed and something is done to try and avoid a recurrence. It's OK to make mistakes, but making the same mistake twice is a bit careless.
The most obvious example of this is making sure that all implementation bugs have test coverage before the issue is considered resolved. However, being proactive matters too. Prior to every Rakudo compiler release, the tests of all modules in the ecosystem are run against it. Regressions are noted, and by now can even be automatically bisected to the very commit that caused the regression. Given releases take place every month, and this process can be repeated multiple times a month, there's a good chance the developer whose work caused the regression will have it relatively fresh in their mind.
A huge number of things have been fixed and made more robust over the last five years, and there are tasks I can comfortably reach to Raku for today that I wouldn't have five years back. Just as important, the tooling supporting the development and release process has improved too, and continues to do so.
There are around 3.5x as many modules available as there were five years ago, which means there's a much greater chance of finding a module to do what you need. The improvement in quantity is easy enough to quantify (duh!), but the increase in quality has also had a significant impact, given that many problems we need so solve draw on a relatively small set of data formats, protocols, and so forth.
Just to pick a few example, 5 years ago we didn't have:
- Cro, which is now the most popular choice for building web applications and services in Raku. Cro wasn't a port of any existing library, but rather designed from scratch to make the most of the Raku language.
- DB::Pg and friends:
while DBIish, which
existed 5 years ago, has become far more mature,
DB::Pgprovides a well-engineered alternative that - at least to me - has an API that feels more natural in Raku. - Red - an ORM for Raku that puts the meta-programming powers of the language to good use. It's marked up as a work in progress, but looks promising.
- LibXML - an extensive Raku binding to libxml native library
- IO::Socket::Async::SSL - yes, at the time of the Christmas release, for all of the nice async things we had, there wasn't yet an asynchronous building of the OpenSSL library.
Given how regularly I've used many of these in my recent work using Raku, it's almost hard to imagine that five years ago, none of them existed!
I was somewhat reluctant to put this in as a headline item, given that I'm heavily involved with it, but the Comma IDE for Raku has been a game changer for my own development work using the language. Granted IDEs aren't for everyone or everything; if I'm at the command line and want to write a short script, I'll open Vim, not fire up Comma. But for much beyond that, I'm glad of the live analysis of my code to catch errors, quick navigation, auto-complete, effortless renaming of many program elements, and integrated test runner. The timeline view can offer insight into asynchronous programs, especially Cro services, while the grammar live view comes in handy when writing parsers. While installing an IDE just for a REPL is a bit over the top, Comma also provides a REPL with syntax highlighting and auto-completion too.
Five years ago, the answer to "is there an IDE for Raku" was "uhm, well..." Now, it's an emphatic yes, and for some folks to consider using the language, that's important.
The Christmas release of Raku was not speedy. While just about everyone would agree there's more work needed in this area, the situation today is a vast improvement on where we were five years ago. This applies at all levels of the stack: MoarVM's runtime optimizer and JIT have learned a whole bunch of new tricks, calls into native libraries have become far more efficient (to the benefit of all bindings using this), the CORE setting (Raku's standard library) has seen an incredible number of optimizations, and some key modules outside of the core have received performance analysis and optimization too. Thanks to all of that, Raku can be considered "fast enough" for a much wider range of tasks than it could five years ago.
Five years ago, Raku wasn't called Raku. It was called Perl 6. Name change came up now and then, but was a relatively fringe position. By 2019, it had won sufficient support to take place. A year and a bit down the line, I think we can say the rename wasn't a panacea, but nor could it be considered a bad thing for Raku. While I've personally got a lot of positive associations with the name "Perl", it does carry an amount of historical baggage. One of the more depressing moments for me was when we announced Comma, and then saw many of the comments from outside of the community consist of the same tired old Perl "jokes". At least in that sense, a fresh brand is a relief. Time will tell what values people will come to attach to it.
With software, it feels like the ideal time to start working on something is after having already done most of the work on it. At that point, the required knowledge and insight is conveniently to hand, and at least a good number of lessons wouldn't need to be learned the hard way again.
Alas, we don't have a time machine. But we do have an architecture that gives us a chance of being able to significantly overhaul one part of the stack at a time, so we can use what we've learned. A number of such efforts are now underway, and stand to have a significant impact on Raku's next five years.
The most user-facing one is known as "RakuAST". It involves a rewrite of the Rakudo compiler frontend that centers around a user-facing AST - that is, a document object model of the program. This opens the door to features like macros and custom compiler passes. These may not sound immediately useful to the everyday Raku user, but will enable quite a few modules to do what they're already doing in a better way, as well as opening up some new API design possibilities.
Aside from providing a foundation for new features, the compiler frontend rewrite that RakuAST entails is an opportunity to eliminate a number of long-standing fragilities in the current compiler implementation. This quite significant overhaul is made achievable by the things that need not change: the standard libary, the object system implementation, the compiler backend, and the virtual machine.
A second ongoing effort is to improve the handling of dispatch in the virtual
machine, by introducing a single more general mechanism that will replace a
range of feature-specific optimizations today. It should also allow better
optimization of some things we presently struggle with. For example, using
deferral via callsame, or where clauses in multiple dispatch, comes with a
high price today (made to stand out because many other constructs in that
space have become much faster in recent years). The goal is to do more with
less - or at least, with less low-level machinery in the VM.
It's not just the compiler and runtime that matter, though. The recently elected Raku Steering Council stands to provide better governance and leadership than has been achieved in the last few years. Meanwhile, efforts are underway to improve the module ecosystem and documentation.
Today, we almost take for granted much of the progress of the last five years. It's exciting to think what will become the Raku norm in the next five. I look forward to creating some small part of that future, and especially to seeing what others - perhaps you, dear reader - will create too.
s/just how far the language/just how far the Raku Programming Language
s/its implementation/its implementation in Rakudo
s/the surrounding ecosystem/the surrounding Raku ecosystem
Was there a JSON::Fast 5 years ago? If not, mention that also? It would be good for some SEO juice I guess. And I know you use it in Cro :-)
s/to write a few line script/to write a script of a few lines/
Feels to me we should have links for "improve the module ecosystem" and "improve the documentation", but I wouldn't know what to link to for that. Do you? If not, maybe just link to https://modules.raku.org and https://docs.raku.org?