draft agenda for modules design meeting

gistfile1.txt

=head1 Rakudo module required features

This is a draft agenda for one of the design meetings planned for the
developers attending the Copenhagen Perl 6 Hackathon on 2010-03-06.

It's all about the C<use>, C<need> and C<require> statements.  Synopsis 11
is the spec.  The new Rakudo must be able to import symbols from .pm files into
the lexical scope containing the C<use> statement, unlike alpha, which
imported the contents into the global scope.

The ultimate goal is to load libraries containing code in a variety of
formats:

=head2 Perl 6

Source code must be parsed and converted to PAST by the Perl 6 grammar,
the same way as with main program scripts, and then its main block run, as well
as any BEGIN-time things fired.

=head2 Perl 5

The only officially correct way to execute a Perl 5 program is using a
standard Perl 5 interpreter.  Therefore Rakudo needs a bridge to connect
via Parrot to a standard Perl 5 installation.  Fortunately both Parrot
and Perl 5 are embeddable.

Blizkost dynamically embeds a Perl 5 interpreter as a library into the
Parrot runtime that is executes Rakudo.

Gabor Szabo wrote a bridge that works the other way round, embedding
Parrot as a library within a Perl 5 process.

There may be another way.  The C<B::C> module cross compiles Perl 5
source code to a C program that runs standalone, and very fast.  It is
already able to run the test suites of frequently used Perl 5 core
modules, and it is very fast.

=head2 PIR

Rakudo/alpha already had this for faster module loading.  The F<.pm>
files are precompiled to F<.pir> using Rakudo's C<--target=pir> and
C<--output=path/module.pir> options.

=head2 PBC

Rakudo/alpha could also use this in a way similar way to PIR modules.
The additional performance gain compared to PIR loading is small.

The bytecode format varies between Parrot versions.  It is very likely
that various Parrot versions will need to exist side by side, freely
sharing data and execution threads.  Multi-parrot execution is probably
not yet implemented, let alone tested.

Hence most users opt for just PIR precompiling.  PBC format will matter
when integrating other Parrot languages that emit code in PBC but not in
PIR format, TCL for example.

=head2 Native machine language

The Parrot NCI gives access to libraries written in C and other
languages.  There are only a few examples of NCI usage, and they have
only been used occasionally from Rakudo, and with difficulty.  What
Rakudo needs is a way to Parrot NCI wrappers at C<use> time, preferably
without having to do a C compile per module.

The Perl 5 XS interface is powerful, but depends on C coding skills and
a compiler to build a wrapper library.  The SWIG alternative is much
easier, but only where it is suitablemay be
is sometimes recommended
It would be great to have something similar to XS, but easier.  In the
Perl 5 development grant applications, schlomifish is proposing a
friendlier design similar to the ctypes in Python.  Can Rakudo have
something similar?

The F<link-c> project had some prototype code.  It mainly marshals
parameters and return values between the Parrot data types and C
structs, and forwards library calls.

=head1 Module storage and versioning

S11 specifies that modules are identified by a hierarchical name, an
author/authority and a version.  A Perl 5 style file system store that
maps the namespace and name to directories and files, cannot distinguish
multiple C<auth> and C<ver> instances sharing a common name.

S11 also allows the C<use> command to request a module whose version is
"best" smart matched 

A modules implementation consists of two parts, name resolution and
storage/loading.

=head2 Name resolution

=head2 Storage/loading

=head1 Distribution

Our starting points are CPAN, the L<proto> installer and L<plumage>.

=head2 Repositories

We use git, subversion, http, ftp, and local folders.