CURLI roadmap ideas, please leave your comments here

gistfile1.txt

The CompUnitRepo::Local::Installation class (or short: CURLI) is a class
that is part of the Perl 6 core for installing modules from any source,
as described in S22.  What S22 does not describe, is directory and file
layout.  I'll try to explain in this post how I think we should do this,
having had the experiences (and problems) with panda and the current
implementation of CURLI.


CompUnitRepo::Local::File
=========================
Technically, the CURLF is not much of a CompUnitRepo, as it doesn't support
installing of modules as such.  It is basically just a frontend to the
module loader that supports Perl 5 like module loading semantics (aka,
without any from / auth / version / api support) on files that happen to be
living at a certain location on a file system.  Therefore, a CURLF should
*never* need to handle precompilation of modules.  A CURLF is intended for
a development situation, *not* for a production environment.


Pluggability / Composability
============================
S11 / S22 describe an API.  That API will need to be further defined as
we get further into implementing CURLI.  The idea should always be that
*anybody* should be able to create their own CompUnitRepo module (think
packagers, or companies, or TPF) to their wishes.  So anything CURLI can
do, should be possible to other developers as well.


CURLI Prerequisites
===================
- modules are installable from a Distribution object.  Typically a tar-file,
as already is being used in the panda ecosystem.  A distribution may
contain 0 or more loadable modules (something a "use", "need" or "require"
can find) and associated meta-information and/or data.
- a distribution contains several types of meta-data.  Some parts of these
will be necessary to select a module for loading (e.g. the auth / version /
api parts needed for the "candidates" method).  Other parts are only needed
when actually loading the module (e.g. the %?RESOURCE hash).  Still other
parts only needed during introspection-like actions at runtime (e.g. calling
.WHY on a code object, if set up in a non-standard way.  Or finding out
the actual textual description of the distribution).
- installed modules need to "survive" a rakudo update/upgrade.  In the
previous panda implementation, modules needed to be re-installed (over
the internet) after *every* rakudo update (and that includes *any* rakudo
recompile).  This becomes very tiresome for core developers and is
therefore one of reasons why some (bad) changes in rakudo are not seen
in the ecosystem until it is too late.
- precompiled modules may need to continue to exist for different versions
of rakudo.  Think switching between different versions of rakudo using
rakudobrew: for a given installed base of modules, you don't want to have
to recompile again and again when switching: you should only need to
precompile *once* for any rakudo compilation.
- precompiling all installed (source) modules should be simple, fast and
possibly asynchronous.  This will allow core developers to more easily do
a sanity check on changes in rakudo.  It could even become part of
"make install".
- the core libraries (such as Test.pm, NativeCall.pm) should be installed
using the CURLI install process.  It's really a matter of eating your own
dogfood.  And in the long run should simplify matters.
- having a large number of modules installed, should have *no* effect on
bare startup time, either for starting the REPL, or doing a -e.
- only when a CURLI is asked to look up a module through .candidates,
should it load the necessary meta-info for satisfying the selection of
a module only.  This should probably be in some file with a dedicated
(rakudo version independent) data-format that contains this meta-info for
*all* installed modules in this CURLI.  For performance, this meta-data
should probably also live in the form of a precompiled hash that would be
much faster to load, as no parsing would be needed.
- the CURLI object for a given base directory, should be a sentinel
object.  It is free to install distributions asynchronously if so required
(perhaps the .install method should allow for a number of distributions to
be installed, instead of just one).  Since the CURLI object is a
sentinel, it can keep all necessary meta-info for selection of a module
in memory, and update both the memory copy, as the one on disk when
installing a distribution.
- the meta-information of a given distribution, can be considered frozen,
as the distribution itself (for a given name / auth / version / api).
Therefore any module related (other than needed for module selection) data
should live in a rakudo version independent format.  With possibly a
precompiled version for performance.


CompUnit prerequisites
======================
- there should only be one sentinel CompUnit object for a given compilation
unit.  Even if two threads are doing a .candidates simultaneously and return
the same compilation unit, both threads should share the same CompUnit
object.
- when rakudo is asked to load a CompUnit object, it will first determine
whether the CompUnit object is already loaded (by checking if its .WHICH is
known).  If it is not known to have been loaded already, it will call the
.load method on the object.  After which the object should be able to
provide pertinent information to rakudo about namespaces created, etc. etc.
(this needs to be worked out further).
- an implementation of CompUnit only needs to be able to .load it: rakudo
should take care of all related issues to prevent circularities and double
loading.


Implementation implications
===========================
- all precompiled code is bound to a specific rakudo version (compilation,
actually).  To make management of rakudo versions more easy for helper
applications such as rakudobrew, all precompiled files that are associated
with a given CURLI should live in a single directory inside the base
directory of the CURLI: removal of a rakudo version (compilation, actually)
would then just mean removing the directory for that compilation.
- S11 stipulates that limitations of the file system (most notably,
case-insensitivity nowadays) should not be able to affect the naming of
modules in any way (so if someone would like to give their module a
Japanese name, that should Just Work(TM)).  This implies some name
mangling from the language visible name, to the actual file on the file
system.  The current CURLI implementation uses numbered files: one could
argue that some SHA might be better.  But since all installation of modules
*should* be handled by a single sentinel CURLI object (even across processes,
so some file system type locking / semaphoring will be needed) it would
seem that simple numbering is adequate, as having SHA's as filenames would
not add much information from a file system (ls) point of view anyway.


Directory layout
================
The rakudo install directory, as well as any CURLI base-directory, may live
anywhere on a file system to which the user has access.  Only executable
components need to be installed in system directories such as /usr/local/bin.

Should one wish to have a global, system supplied rakudo, only then it seems
warranted to actually have the rakudo install directories, as well as any
CURLI base-directories, at a system location, under protection of sudo/root.

rakudo install directory 
 |
 \-- .precomp
       |
       \-- (compilation ID: one for each SHA of rakudo)
             |
             |-- perl6.moarvm (BOOTSTRAP)
             |
             |-- CORE.setting.moarvm
             |
             |-- RESTRICTED.setting.moarvm
             |
             |-- installed modules meta info (needed for .candidates)
             |
             |-- (distribution ID: one for each installed distribution)
             |     |
             |     \-- (compunit ID: one for each compunit in the dist)
             |           |
             |           |-- runtime meta info if any (e.g. %?RESOURCE)
             |           |
             |           |-- precompiled file
             |           |
             |           \-- other precompilable data
             |
             \-- lib
                   |
                   \-- (compunit ID: one for each of Test, NativeCall, etc.)
                         |
                         \-- precompiled file


(base-directory: one for each CURLI)
 |
 \-- (distribution ID: one for each installed distribution)
       |
       |-- module meta data for .candidates
       |
       \-- (compunit ID: one for each compunit in the dist)
             |
             |-- runtime meta info needed for loading
             |
             \-- .dist
                   |
                   * original distribution files name mangled


Cleanup considerations
======================
- removing support for a compilation ID of rakudo:
  rm -rf rakudo install directory/.precomp/(compilation ID)
- uninstalling a distribution ID:
  rm -rf base-directory/(distribution ID)
  rm -rf rakudo install directory/.precomp/(distribution ID)


Rakudo loading process (on MoarVM)
==================================
- "perl6" is a script that loads moarvm in the install/bin directory.
  - it passes the name of the script as execname
  - it passes a libpath to the nqp/lib
  - it passes a libpath to .
  - specifies perl6.moarvm (main.nqp) to be run with the given parameters
    - this loads nqp modules Perl6::Grammar & Perl6::Actions
      - Perl6::Grammar loads Perl6::World
    - sets up a Perl6 compiler
    - calls it with the given (and generated) parameters
      - token comp_unit then loads the settings (CORE.settings.moarvm)
    - runs the indicated code
    - runs the END blocks

From -use- to actual loading
============================
- the use/need/require statements should simply generate code that will
call the appropriate settings sub (e.g. USE/NEED/REQUIRE).  This will move
most of the higher logic of compunit loading to the Perl 6 level, where it
is more easily maintained.  Since the actual loading for standard compunits
will still happen at nqp level, the performance consequences should be
minimal.
- to further facilitate development and maintenance, the USE/NEED/REQUIRE
should probably be multis, with at least different candidates for :from.
This should allow loading of compunits to become even more pluggable,
provided a different :from is used.  Such a candidate could even be
exported by a module (think auto-generated code from a WDSL template
that you could simply access by saying 'use FooBar:from<wdsl>, where
the FooBar file would contain the indicated template).
- the default implementation of USE/NEED will simply go through @?INC
and ask for .candidates to each of the CUR's.  The first CUR to return
any CompUnit objects, will stop the search.  If more than one CompUnit
is returned, a tie-breaking mechanism will be employed.  If this does
not result in only one remaining CompUnit, then error should occur
(think a class consuming two roles, where each role is supplying a method
with the same name: the class will need to resolve this by supplying its
own method with that name).  In the case of more than one CompUnit, the
issue should be resolved by supplying a stricter -use-/-need- statement.
- the tie-breaking logic of USE/NEED can be governed by pragma's in the
future.  For now, the tie-breaking logic is as follows: if the CompUnits
are of different auth's, the tie-break will fail.  If the CompUnits have
different "api" values, the tie-break will fail.  If all CompUnits have
the same "api" value, then the CompUnit with the highest version will
be selected.

For maintaining different versions of rakudo, I wonder if we can borrow something from portage — SLOTs and eselect came to mind when reading that portion (shows what system I use, doesn't it? :P).

Instead of 'rakudo install directory', I'd recommend somewhere within that for .precomp; since I install my rakudo to /usr/local, as I understand it that would mean /usr/local/.precomp for precomp, which I'm sure is generally a Bad Idea™. /usr/local/share/perl6/ (or even /usr/local/lib/perl6/) come to mind as possible locations, or more generally $PREFIX/{share,lib}/perl6/ Maybe then you could avoid the reason for hiding the name with a leading dot, even ☺.

Can the mentioned "base-directory" directories be theoretically anywhere on the system, or are they meant to be anchored somewhere? The rm -rf example for uninstalling a distribution ID seems to suggest a "free" dist ID directory, but it's not clear.

The SHA1 name mangling sounds reasonable to me. The only reason I'd suggest a more human-readable filename mangling is in case you need to poke at a specific precompilation (for whatever reason), it'd be far far easier to find if the mangled name resembled the original. With that approach, you could also tailor the mangling to the system, which may save some work on systems with more capable filenames.

Finally, I'd suggest some sort of "auto cleanup" for when you're constantly updating your bleeding-edge copy of rakudo. In that case, you likely don't want to keep precomp from a commit of rakudo you'll never checkout again (as opposed to rakudobrew's use case). However, I could easily see that being the domain of some external tool, if more appropriate.

• does this mean that i'm open to putting whatever i want in a CURLI manifest file?
• how does CURLF store information about multiple modules with :auth and :ver, it doesn't seem like it's that difficult a problem.
• how do i handle not being able to write to a precomp directory, is that a total failure for installing the module?
• this doesn't seem to imply that a CURLF and CURLI should have the same interface, does the loader behave differently depending on which one he's calling .candidates on?
• is it the CURL{I|F}'s job to recomp modules when perl6's version changes?
• are old version precomp'd versions of modules removed when the version changes or left on the FS (regardless of which component is doing the precomp)?
• which .candidates should be loaded if there are many (multiple Text::CSV, Tux made 3 of them for fun) returned from a .candidates, first available, highest version str, most recently installed?
• if precomp fails should installation fail of a new module fail, it doesn't seem like it should because then a package manager can then prompt and install anyway or fail.

EDIT for shimmerfare

• this is out of scope for this discussion
• this is also out of scope
• sha1 mangling and supporting multiple author/ver/auth means maintaining a MANIFEST, at that point nothing about the ordeal is human readable. and half mangling something seems a lot like chasing a fairy
• i don't think auto cleanup is the right way. if i'm a developer or even someone who may have to revert perl6, i don't want my precomp's to go away.

Will precompilation be purely an installation action, or do we want to precompile modules at load-time, too if we detect the precompiled version to be outdated? In the latter case, we need to store the precomp files in a directory where the user is actually allowed to write to. If we precompile only on installation, $PREFIX/lib/perl6/ would be ok.

niner: Because a distribution is assumed to be immutable, I see precompilation purely as an installation action to be done at installation time. At least, the precompilation done by CURLI. I'm not 100% sure what precompilation for CURLF would mean, but I guess even in a development environment, one could benefit from JIT-compilation. But for CURLI, it would be just for installation.

Shimmerfairy: apart from directly executable programs (such as perl6 and perl6-debug), I don't think anything would need to live in a system directory such as /usr/local/. So, if I mention "rakudo install directory", it means a directory somewhere on the system where the installer has write access to. If the installer is an admin, I would think some kind of sudo access would be needed. If the installer is a normal user, anything within the user's home dir should be fine. P5's local::lib functionality should be possible from the start, without anything needed to be bolted on later.

Regarding name mangling and access to precomped files: you probably should never want to do that from the file system's point of view anyway. If you really want to know which module's precomp lives where, the meta-info should be able to tell you. And a small helper script using the CURLI API should be able to tell you quickly where it lives, and allow you to inspect it.

tony-o: to answer your questions, as far as I can:

• you can put whatever you want in the META6.info: it only defines required and optional fields. Any other fields will be ignored, but should be introspectable at runtime later.
• CURLF is for development only: it ignores any :auth / :version settings
• if you can't write the precomp dir, it's an install failure
• the loader loads something by calling the .load method on a CompUnit object. calling .candidates on a CURLF or CURLI object, should provide you with 0 or more of them. CompUnit objects however, could be made separately if necessary. Any object that provides the CUR API, can return CompUnit-like objects in any shape or form, as long as the .load method does what it is supposed to do to load that compilation unit.
• CUR objects should just provide CompUnit objects (through .candidates). The CompUnit object should have a .precomp method for precompilation (which could be a noop in the CURLF, e.g.). Re-compiling all installed modules would: call .candidates on all CUR objects in @?INC using no filter whatsoever, and then call .precomp on all the CompUnit objects provided.
• removal of precomp files should be part of the removal of a version of rakudo from the system (see Cleanup considerations).
• since the visibility of a loaded module is always in a lexical scope, and you can't have two things with the same name in a lexical scope, finding more than one CompUnit by .candidates in a single CUR, should be considered an error. If no CompUnit's are found, it should go to the next CUR in @?INC.
• if a precomp fails during installation, a CUR is free to decide what to do with it. In the case of CURLI, I would think it would need to fail. Other CUR's might decide differently. Whether package managers will use CURLI to install modules, is still open to debate: the "installed modules meta info (needed for .candidates)" needs to be updated during install time, so it won't work by just copying files. On the other hand, something like an AppPacker, FatPacker could very well create a CURLI for a specific app, containing all of the modules it needs, and just run Perl 6 with only that single CURLI.

CompUnitRepo::Local::File

Technically, the CURLF is not much of a CompUnitRepo, as it doesn't support
installing of modules as such. It is basically just a frontend to the
module loader that supports Perl 5 like module loading semantics (aka,
without any from / auth / version support) on files that happen to be
living at a certain location on a file system. Therefore, a CURLF should
never need to handle precompilation of modules. A CURLF is intended for
a development situation, not for a production environment.

I disagree. There will be situations, maybe even in production, where you want to install to a CURLF.
There should be always more than one way to do things, and allowing to install into a CURLF should be as possible as copying some code from panda to rakudo. That said, CURLF should support precompilation but it should not, or rather cannot, support auth/version.

since the visibility of a loaded module is always in a lexical scope, and you can't have two things
with the same name in a lexical scope, finding more than one CompUnit by .candidates in a single
CUR, should be considered an error. If no CompUnit's are found, it should go to the next CUR in
@?INC.

Again I have to disagree. If your filter for looking up dists is lax and allows a range of potential dists, the it is it a decision which we have to respect. If I want to accept Text::CSV 1.2.+, and v1.2.0 and v1.2.5 is available in the first CUR with a result, I expect that v1.2.5 is picked. This picking happens when a CU is compiled. I.e. every time I load a .pm source file, or at precomp time of a .pm.moarvm.

As for the directory layout of installed dists. I think a combination of numbers and the (almost) original filename will be acceptable. "Almost" means something like punycode, where only non-ascii filenames are mangeld, whereas others stay intact. see https://github.com/FROGGS/p6-IDNA-Punycode/blob/master/README.md

So:

$PREFIX/0/bin/bubble-breaker.pl
$PREFIX/0/lib/Games/BubbleBreaker.pm
$PREFIX/0/lib/Games/BubbleBreaker.pm.moarvm
$PREFIX/0/resoure/images/blue.png
$PREFIX/0/resoure/images/green.png
$PREFIX/0/resoure/images/red.png

Though I understand the reasoning about the precomp'd files and special directories for cleaning them up more easily.

When rakudo tries to rebuild/install its modules when updated, how does it:

• Run any hooks that take place during various install phases? Is CUR::* supposed to provide the hookable locations/phases? How does it re-run the hooks unless the hooks are installed?
• Should CUR::* be providing bin shims for backends that are not installed?
• How does it test the modules to make sure they work with the upgraded rakudo? That the module even works with any possibly upgraded dependencies that were installed after the original module installation?

If CUR::* is responible for precompilation, how does it:

• Handle cross precompilation for different backends?
• Handle a failure of one backend's precomilation but not the other?
• Allow installation of source only for certain bugs that may present themselves only when precompiled for a certain backend but otherwise work OK when run from source?
• Allow the user to supply the precompiled modules themselves? Allow the user to supply only a precompiled version of the module with no source?
• Determine the build order of the provides?
• Handle the build order of the external dependencies? What if this changes based on optional dependencies that get installed later?

How does Distribution.content fit into this scheme?

Is there still supposed to be a MANIFEST (not referring to META6.json)? If so, shouldn't this be involved in the uninstall process? Or if each package is meant to contain its meta information in its META6.json in its own folder, would this slow down the loading of modules further by requiring more files to opened?

Where should CUR::* alternatives be installed to so they can be loaded with rakudo? Wouldn't implementing and specing this first be an ideal first step, so alternative CUR::* implementations can be presented and tested without having to bake them directly into rakudo and requiring whomever to compile and run a seperate branch? I think so, its why I stopped working on it.

If uninstall just does an rm -rf, what happens to everything that depended on it? Is that now the responsibility of the user, or should rakudo also handle orphaning?

FROGGS: We agree on CURLF not supporting auth/version. One should also realise that naming of compunits in a CURLF is limited by the filesystem capabilities: on a case-insensitive file system, one could not have a "foo.pm" and an "Foo.pm" installed. One would also be limited by directory entry length or code pages. I think this is fine for a development situation, but not for production.

With regards to requesting a Text::CSV (without auth), suppose we have one by Tux and one by me. Which one should it choose then? In my opinion, returning more than one CompUnit by .candidates, is the same situation of a class consuming two roles each supplying a method "foo". The developer will need to resolve this conflict at development time. This is the same for loading compunits, in my view. And resolving this problem, is by making sure that you are strict in what you want. That is the only way out of dependency hell that has been plaguing Perl 5 and other languages as well. By saying which version of a compunit you need in a lexical scope, you are ensuring that your code will always run in the future as long as that version of the compunit is installed on the system. And that is a matter of dependency management, which is another thing altogether.

I would even go as far as not allowing an (under)qualified -use- statement in production code (or at least warn about it). Underqualified -use- statements are a disaster waiting to happen, as history in Perl 5 has shown (since Perl 5 only as unqualified -use- statements).

With regards to directory layout: since we want to keep multiple precomped versions of modules around, they simply cannot live in the same directory of the source file (unless we start encoding the compilation ID in the filename or extension, which feels like a unneeded dirty hack). So it's not only about being able to clean up quickly, it is also about being able to keep multiple versions around.

ugexe: trying to answer your questions

• each CUR implementation can decide on which hooks it supports, and how they are specified. I agree CURLI should have support for hooks of many steps in the installation process. Further than that notion, I have not come yet. I hope we will be able to finalize this in this round of development. If you have any ideas in that direction, I would like to see them (maybe in a separate gist?)

• like Perl 5, I think Perl 6 can only install compunits for the currently running process. Which implies that a backend is already selected. And that also implies to me, it should only install any files only needed for that backend. If you want to be able to run it on another backend as well, one would need to install the compunit using that other backend. This could maybe be automated by having one Perl 6 shell out to another Perl 6 using a different backend. But I would think a process can only install for the backend it is running on itself.

• the complete original distribution, along with test files, is installed on the system, in my view. This should allow for rerunning any tests, should one so desire. But I would definitely not expect that to happen by default when upgrading a rakudo. With regards to upgraded dependencies: if you are stupid enough to run with underqualified -use- statements in production code, you deserver what you get. Perl 6 gives us the unique opportunity to get away from dependency hell. It only requires a little strictness from developers: if you are sure that the code in a given lexical scope runs with version1.2 of Foo, then specify that in your -use- statement. Then you will never have to worry about upgrading rakudo, as long as the that compunit remains installed. And that is a matter of dependency management.

• CURLI can only precomp for the backend it is currently running under. Technically, it probably could precomp for other backends (by shelling out to a Perl 6 with a different backend). But I would, at least for 6Mas, disallow that for now, and possibly for ever.

• since CURLI can only precomp for its own backend, it only needs to handle failures in that case.

• with the proposed directory layout, it should be possible for different backends to use the same CURLI base directory. So a CURLI should probably understand that its sibling in the same base directory, but on a different backend, already installed the distribution files. It would still need to update its precompiled version of the installed modules meta data (needed for .candidates). But since the rakudo on the other backend will have a different compilation ID, the precompiled meta info will live in a different directory, so they will not interfere with each other between backends.

• user supplied precomped files, even without source, smells like a package manager wanting to control things. This is fine, but I don't think the standard CURLI needs to support this. The whole point of CUR is that anybody can make their own CUR implementation, as long as it provides CompUnit objects that can be .load ed. So if a package manager want to just supply precomped files, they would have to make their own CUR (or find someone in the P6 community to make one for them).

• we will need a way find out the build order of provides. Suggestions welcome.

• we will need to look for prior art in this matter. Suggestions welcome.

• I'm not sure what you mean with Distribution.content.

• we may want to keep a MANIFEST around. On the other hand, since we try to keep things that belong together, together as much as possible, and prevent files from being copied at all, there is much less to clean up.

• the parts of META6.json that are needed for .candidates, are kept separately in a fast readable format already. The parts of META6.json that are needed to load a compunit, are also kept separately in a fast readable format. The original META6.json file will be introspectable on demand: only then should it be opened. So not for selecting the compunit, or for loading it.

• the location of a non-core CUR, is indeed an issue. I think we will need a separate command line parameter (and/or environment variable) for this, along with specification of its "source". I was thinking something along

  perl6 -C /usr/lib/CloudPan.pm -I cloud#cloudpan.org/6pan

The -C would take a path to a CUR (possibly even precompiled), which would get loaded (because it would generate an on-the-fly standard CompUnit object) and run its BEGIN blocks. This would install the "cloud" prefix for -I . With the -I specification, you would then instantiate a CloudPan CUR and unshift it on @?INC. And the you would be in business. :-)

• you should not uninstall modules unless you are sure that no other modules need it, and none of your production code needs it. This is no different from the current situation in P5 or other languages. OTOH, I think uninstalling, with current disk size to distribution size ratio, is something that we will need to worry about less and less. If you really are short on disk space, maybe we need to support some sort of generic precomp only CUR. Or maybe someone out there will just make one, because the can!

Some notes on your notes:

Installation of precompiled files only with no source was your suggestion, it has nothing to do with a package manager: http://irclog.perlgeek.de/perl6/2015-05-27#i_10665828 Additionally a user may wish to supply their own precompiled files because precompiling on things like a raspberry pi on the JVM is incredibly slow.

Distribution.content: http://design.perl6.org/S22.html#content

Regarding tests, S22 said tests would not be installed which is why I brought it up. http://design.perl6.org/S22.html#t

The definition of underqualified use statements does not seem as simple as I interpret from your comments due to supersedes, superceded_by, and excludes from S22 combined with later installation or uninstallation of modules.

ugexe:

• then we may need to have a CUR for installing precomps only in the ecosystem. I don't see that as something that would need to be supplied by the core.
• ah, that Distribution.content! :-) The idea of that was, that if we have support files in a Distribution, the file names will be mangled on the file system. The .content method should provide the programmer with a transparent way to get at those files, by specifying their "logical" location, instead of their actual location on the file system (which may be unknown before actual installation of the distribution).
• I guess the thinking on that has changed: the entire distribution will live unpacked, but name mangled, somewhere under the base-directory of the CURLI. Precompilation of compunits will happen directly from the files in that directory, into the appropriate directory under .precomp. Since precomped files under rakudo refer to each other by absolute paths (afaik), this is another reason why putting them together may actually be a performance win (fewer disk reads necessary).
• supersedes and superseded_by & excludes influence the selection process in .candidates, nothing else. So suppose you have use Foo:ver<1.2>:auth in your code, and it has been superseded by 1.21, then .candidates will return the CompUnit for 1.21, instead of the one for 1.2.

Regarding precompiled files only:

No other CUR is needed for installing precompiled only code. CompUnit is what currently prevents this with the way it handles initialization using assumptions about the path passed in. The problem is for it to work at all you would also need to allow external CompUnit implementations to be used. It seems trivial to just allow a CompUnit to work in this fashion in the core (by looking at the extension like is already done for the rest of CUR instead of the current method of assuming $!path is a source file and just tacking an extension onto it and declaring that the precompiled path [like File.moarvm.moarvm]) vs requiring both an external CUR and CU to be supplied on the command line.

Regarding supersedes:
S22 says it has additional meaning for external packagers, so it's effects would go beyond .candidates

By saying which version of a compunit you need in a lexical scope,
you are ensuring that your code will always run in the future as long
as that version of the compunit is installed on the system.

That is also begging for letting it fail later... You forgot that the system you are running on is not something constant. It will change over time, and when that happens you want to pull in a patched dependency, or a patcherd dependency of a dependency. This is the reason why the v1.2.+ sytanx exists, and you (the compiler) is not in charge of disallowing its use.
The problem with the dependency of a dependency situation is that you often cannot manipulate the dependency. So you cannot change it to allow a new very specifc version. This means you want to have depencies that allow a range of versions for allowing to get bugfixes in.
The same is what you want for your software. You want to constrain module use of a given auth or auth set, but constrain the version down to a range that allows bugfix releases.

FROGGS: I realise that the system you are running on is not something constant. And that you want to prevent use of a buggy version of a compunit. For that, we have "supersedes": it will allow you to specify an alternate compunit B to be used whenever something tries to load a faulty compunit A. I think this is a much better mechanism then relying on version numbers / semantic versioning.

And if it is really a minor version bump with bug fixes and completely identical API, you cannot even remove the lower versioned compunit from the installation. Because you cannot guarantee that there isn't code in your installation, that has a fixed dependency on the lower numbered version.

So I really think the superseded mechanism, is a much better choice. Note that this only affects the selection process in .candidates of CURLI: if a compunit is initially selected, but supersede information is available for it, instead the other compunit will be used (unless there is supersede information for that available as well, of course).