Thoughts on implementing S11/S22 wrt to loading modules

gistfile1.txt

Thoughts on loading modules:

- @*INC is dead, longe live @?INC   **** IMPLEMENTED ***
The places to look for loadable modules, are a *compile time* setting, so it
should be set at compile time.  Hence it should be called @?INC.

- use lib is a lexical pragma    *** IMPLEMENTED as "cur" atm ****
At the moment, use lib just sets @*INC and therefore has action at a distance
effects.  Now that we have a lexical pragma framework (under the hood), I can
make "lib" truly lexical.

- initializing @*INC/@?INC is too expensive  *** IMPLEMENTED ***
Indeed it is.  Therefor I'm taking inspiration from Perl 5, where @INC is just
an array of strings.  In Perl 6, @?INC will also just be an array of strings
in INCLUDE-SPEC format (e.g. file#/foo/bar or inst#/installed/modules or
cloud#cpan for a custom CompUnitRepo format) that can be cheaply initialized.

- pragma-like modules should be installed as any other
Modules such as Test / NativeCall should be installed with a CURLI, rather
than with a CURLF.  As an intermediate step during the migration period from
@*INC to @?INC, these modules should be loaded bypassing the @INC mechanism
as if they were in memory pragma's.

- loading a (non-pragma) module  *** IMPLEMENTED without visible %?INC just yet ***
Only if it is needed to load a (non-pragma) module, will the @?INC array be
searched.  Each string is first absolutified by calling the "absolutify" method
on the class associated with the short-id of the string (if there is any such
method).  This allows for a generic solution to e.g. "file#." as an INCLUDE-SPEC
so that the CompUnitRepo depends on the current setting of $*CWD.

The resulting string is then used as a key in a global %?INC hash, which
contains the associated CompUnitRepo object (or has one created with the
right type if there is none yet).  This will lazify CompUnitRepo initialization.

- searching
Once a CompUnitRepo object has been obtained, the .candidates method will be
called for the given shortname and named parameters.  If no candidates are
returned, the next CompUnitRepo object (deduced from @?INC) will be tried.
If more than one candidate is found, a "multiple candidates found" compile
time error will be thrown.

If one candidate is found (which is a CompUnit object), it will be loaded
using its .load method.  This "load" method itself will be responsible for:
  - checking if already loaded
  - handling recursive loading
  - any other now obfuscated issues in NQP
  - setting $?DISTRIBUTION, %?RESOURCES, etc. in its mainline
This will allow for complete pluggability for any additional type of loading
of compunits in the future (think CloudPan, from distributions, or even from
a database).

Please note that a CompUnit object can be created by any means.  This should
allow handling run-time cases.

- depth-first vs breadth-first
An alternate way of searching for modules, would be to *always* search *all*
active CompUnitRepo objects, before potentially finding more than one
candidate (and erroring out then).  THe most important reason for searching
depth-first, is in any development situation, where a developer needs to
force new versions of modules to be loaded *before* finding any modules
with the same name later in the @?INC path.  Secondly, on large install
bases, search *all* CompUnitRepo objects (especially the ones that may
need to query an external data source, such as a database), may be
prohibitive from a performance point of view.

The argument that maybe the wrong version of a module would get loaded,
is a problem of the developer, really.  If a "use" command is
quantified by from/auth/version, and it still doesn't load the right
version (because of a match in an earlier CompUnitRepo), I would argue
that the quantification of the from/auth/version is insufficient: with
the given quantiification it shouldn't matter which one was selected.
If it *does*, than the selection criteria weren't precise enough,
and the developer should change the "use" command.

Therefore, I think that depth-first is a better approach than
breadth-first.

If a "use" command is quantified by from/auth/version, and it still doesn't load the right
version (because of a match in an earlier CompUnitRepo)... the developer should change the "use" command.

What's the user (as against developer) story? Might there be ways in which apps work fine after installation using CURs meant for end-users (not devs) -- and then stop working when the user upgrades apps or installs new modules?