gistfile1.md

Rakudo and NQP Internals Workshop

Here is an initial proposal for the workshop. It suggests an intensive 2-day workshop taking a deep dive into many areas of the Rakudo and NQP internals, mostly focusing on the backend-agnostic parts but with coverage of the JVM and future MoarVM backends too. While the overall focus will be on getting to know the Rakudo and NQP internals, we'll also build the odd small langauge and object system along the way to get a better handle on the primitives.

Prerequisites

A reasonable knowledge of the Perl 6 language and, preferably, a little experience working with NQP also.

Day 1

The eagle's eye view: Compilers, and the NQP/Rakudo Architecture

• What compilers do
• What runtimes do
• Perl 6 challenges
• NQP as a language
• NQP as a compiler construction toolchain
• QAST
• The nqp:: op set
• Bootstrapping in a nutshell
• How Rakudo uses NQP

The NQP Language

• Design goals
• Literals, variables, control flow
• Subroutines, pointy blocks, closure semantics
• Classes, attributes, methods
• Regexes and grammars
• Roles
• Multiple dispatch
• Built-ins and nqp:: ops
• Exception handling
• Limitations and other differences from full Perl 6
• Shortcomings

The compilation pipeline

• The HLL::Compiler class
• Frontends, backends, and the QAST between them
• Parsing with grammars, AST building with actions
• Code generation
• Building a tiny language from scratch

QAST

• QAST::Node, the base of it all
• The overall structure of an AST
• Literals: QAST::IVal, QAST::NVal and QAST::SVal
• Operations: QAST::Op, basic examples
• Sequencing: QAST::Stmts, QAST::Stmt
• Variables: QAST::Var, QAST::VarWithFallback, scope, decl, value
• The block symbols table
• Invocation
• Parameters and arguments
• Contextualization: QAST::Want
• Block references: QAST::BVal
• Object references: QAST::WVal
• The backend escape hatch: QAST::VM
• At the top: QAST::CompUnit

Exploring nqp:: ops

• Arithmetic
• Relational
• Aggregate
• String
• Flow control
• Exception related
• Context introspection
• Big integer

Day 2

6model

• Objects: behavior + state
• Types and kinds
• Meta-objects
• Representations
• STables
• knowhow, the root of it all
• Building a simple object system from scratch
• Adding objects to our little language, using a World
• Method caches
• Type checking
• Boolification
• Invocation
• Exploring NQP's meta-objects
• Exploring Rakudo's meta-objects
• Container handling

Bounded Serialization and Module Loading

• The compile-time/runtime boundary
• Serialization contexts
• QAST::WVal revisited
• What's "bounded" about it
• Repossession, conflicts and other such terror
• nqp:: ops related to serialization contexts
• The World, revisisted
• How module loading works

The regex and grammar engine

• The QAST::Regex node and its subtypes
• Cursor and Match
• The bstack and the cstack
• NFAs and Longest Token Matching

The JVM backend

• An overview of the JVM
• The QAST to JAST translator
• The runtime support library

The MoarVM backend

• An overview of MoarVM
• The QAST to MAST translator