For a long time, I've been fascinated by strange loops.
The concept "strange loop" revels in the circular definition. Something is the way it is because it's defined in terms of itself. I think fascination for this concept is related to fascination with the infinite. Chicken-egg conundrums. Grandfather paradoxes. Catch-22s. ("Catches-22"?) Newcomb's paradox. Gödel's incompleteness theorems. Something about causal feedback loops tickle our brains.
The concept has taken root not just in art or in brainteasers. When we talk of "bootstrapping compilers" or "metacircular evaluators", we mean exactly this type of activity. To make a compiler bootstrapping means to make it able to compile itself by (re-)writing it in the target language. An evaluator is metacircular if it implements features not in terms of a helper subsystem, but in terms of themselves. And people do this not just because it's a cool trick, but because it opens up possibilities. Arbitrary limits in programming stop us from doing what we want, and the limit between "host system" and "guest system" is arbitrary.
In other words, if you're seriously interested in extensibility, then you're also likely interested in strange loops. That's why you see these ideas floated around in the Lisp communities a lot. And that's why compilers sometimes take the bootstrapping route. (Another happy example is JavaScript, where some language extension proposals happens through their metacircular interpreter, aptly named Narcissus.)
Perl 6 has three strange loops. Grammar, OO, and program elements. It's in various stages of closing those.
Status: not quite closed yet
Perl 6 grammars are famously powerful and allows the programmer to specify entire languages. It stands to reason that a Perl 6 implementation would want to eat its own dogfood and use a Perl 6 grammar to specify the syntax of the language itself. And behold, Rakudo and Niecza do exactly that. (Pugs doesn't, but among its future versions is the milestone of a self-hosting grammar.)
Having this bootstrapping in place gives us a high confidence in our grammar engine. We know that it's powerful enough to parse all the Perl 6 we pass it every day. This is also what allows us to write most of Rakudo in Perl 6, making it easier for others to contribute.
I say "not quite closed yet" because the access from user code into the Perl 6 grammar has some ways to go still.
Benefits: tooling, slangs, macros
Something like PPI should be trivial once we close the loop completely. The compiler already has access to all that information, and we just need to get it out into user space. Slangs can either extend the existing Perl 6 grammar, or start from scratch and provide a completely different syntax. In either case, good communication between old and new grammars is required for e.g. variables to be visible across languages. Macros need to be able to effortlessly go from the textual form of something to AST form.
Status: closed
The system underlying all objects in the Perl 6 runtime is called 6model. An object was created from a class, a class was created from a "class metaobject". After just a few more steps you end up with something called a "knowhow" — this is the end of the line, because by all appearances the knowhow created itself.
6model is our object-oriented implementation of object orientation. It's very nice and it allows us to extend OO when we need it.
Benefits: make your own OO rules
You will find most examples of the kind of OO extension that's posssible through jnthn's modules. Grammar::Debugger extends what it means to call a grammar rule. Test::Mock similarly keep track of how and when methods are called. The recent modules OO::Monitors and OO::Actors both provide new class metaobjects, basically giving us new rules for OO. They also go one step further and provide definitional keywords for monitors and actors, respectively.
Status: just discovered
Macros can generate code, but in some cases they also analyze or extend code. That's the idea anyway. What's been stopping us so far in realizing this in Perl 6 is that there hasn't been a standard way to talk about Perl 6 program elements. What do I mean by "program element" anyway? Let's answer that by example.
Say you have a class definition in your program. Maybe you have a fancy editor, with refactoring capability. The editor is certainly aware of the class definition and can traverse/manipulate it according to the rules of the language. In order for it to do that, it needs to be able to represent the class definition as an object. That object is a program element. It's different from the class metaobject; the former talks about its relation to the program text, and the latter talks about its relation to the OO system.
<PerlJam> masak: "program elements" reads like "DOM for Perl" to me.
<masak> yep.
Macros are headed in the same way as such a refactoring editor. By handling program elements, you can analyze and extend user code at compile time inside macros. The API of the program elements give the user the ability to extend the Perl 6 compiler itself from library code and user code.
The work on this has only just started. My progress report so far is this: Perl 6 has a lot of node types. :) Having a rich language that doesn't take the Lisp route of almost-no-syntax also means that the API of the program elements becomes quite extensive. But it will also open up some exciting doors.
Benefits: macros, slangs
Part of the reason why I like Perl 6 is that it has these strange loops. Little by little, year by year, Perl 6 lifts itself up by the bootstraps. There's still some work left to close some of these loops. I've been hopefully waiting for a long time to be able to do Perl 6 parsing from user code. And I'm eager to provide more of the program element loop so that we can write Perl 6 that writes Perl 6 from inside of our macros.
Mostly I'm excited to see people come up with things that none of us have thought of yet, but that are made possible, even easy, by embedding these strange loops into the language.
Have a loopy Christmas.