patrickbkr/second_try.md

## gistfile1.txt
What Python does
================


See `Python-3.7.0/Modules/getpath.c`. Python always determines its library locations at runtime (prefix and exec_prefix).

It first determines the executable location (robustly, including $PATH and symlink handling), then performs several methods to determine `prefix` and `exec_prefix`, two paths that point to library and shared object folders in the
installation.

1. Check if it's run from the build directory (by looking for a marker file). Use a separate path layout matching the build directory in that case.
2. Look at the $PHTHONHOME environment variable. If it's there, use it.
3. Search for the installation path relative to the executable location (climb up the folder hierarchy and check every folder encountered for a marker file).
4. Use the hard coded preprocessor variables.

Reasoning of the above approach (just guessing):

1. Build dir path allows running a freshly compiled python during the build phase.
2. The environment variable to allow overrides.
3. This is what makes relocatability possible.
4. This is described as a last fallback that is usually not helpful. But I think it is what most distributions would want to use in a fixed installation scenario.

I think a better ordering of the above checks would be:

1. Check for the environment variable.
2. Use the hard coded variable if it was set in the build.
3. Search relative to the executable.
4. *Don't* implement the build dir check. If that's needed we can simply use the custom runners we already have.

That way a build is either relocatable (with the hard coded variables unset) or located (with the variables set).

Skipping the build dir checks prevents some potential funky attack scenarios (place an installation in / or something) and we do not pay for the search. Setting the hard coded variables to empty just skips the search altogether.

## second_try.md

      
    Raw
  

              second_try.md
            
          
    The goal

Make perl6 relocatable. So when moving a folder with a moarvm, nqp and perl6 installation inside around, it will still work.
What Python does

I looked at how python does it. They look for files relative to the running python executable.
Then they check for one of four conditions and acts accordingly.

Check if it's run from the build directory (by looking for a marker file). Use a separate path layout matching the build directory in that case.
Look at the $PHTHONHOME environment variable. If it's there, use it.
Search for the installation path relative to the executable location (climb up the folder hierarchy and check every folder encountered for a marker file).
Use the hard coded preprocessor variables.

What Perl6 currently does


The installation prefix is given at configure time and compiled into the executables.
Runner scripts (bat/sh) are generated. Different ones in the build directory and installation directory.

In the build dir --libdir contains ".", "blib" and "$NQP_INSTALL_DIR/lib"
Installed scripts contain --libdir="$PERL6_INSTALL_DIR/lib", "$PERL6_INSTALL_DIR/runtime" and "$NQP_INSTALL_DIR/lib"


The build dir runner scripts also give an --nqp-dir="blib". That is used to locate the BOOTSTRAP.moarvm file.
CompUnit/RepositoryRegistry.pm6 uses the installation prefix path that was compiled in and uses it to set up its three repos: "/", "/site", "/vendor"

My first approach

Do it similar to python:

Check for two environment variables.
Use hard coded variables if they were set during the build.
Search relative to the executable and derive the values the environment variables would have.
(4. Don't implement a build dir check. We can simply use the custom runners we already have.)

The two environment variables would be:
PERL6_HOME=/share/perl6
NQP_HOME=/share/nqp

from these the following locations are derived and used:
libpaths = $NQP_HOME/lib, $PERL6_HOME/lib, $PERL6_HOME/runtime
main = $PERL6_HOME/runtime/perl6.moarvm (or perl6-debug.moarvm)
comp-repos = $PERL6_HOME, $PERL6_HOME/site, $PERL6_HOME/vendor

In the usual case the environment variables are unset and the locations are derived from the executable files location as follows:
PERL6_HOME=$EXEC_DIR/../share/perl6
NQP_HOME=$EXEC_DIR/../share/nqp

Problems with this approach

Distributions like to order stuff differently, e.g. SuSE does that. They put the
NQP libs in /usr/lib64/nqp, without a /lib subfolder. So NQP_HOME can not work
anymore because the lib subfolder is missing. It is probable something similar
will happen to the files in $PERL6_HOME/runtime.
The same is true for the folder layout in the build directory. There the
compiled files are either located in "/" or in "/blib". That can be worked around
by generating different runner scripts in the build directory (basically just use the ones we already have).
My second approach

We have three environment variables:
RAKUDO_BASE_LIB=/share/nqp/lib,/share/perl6/lib,/share/perl6/runtime
RAKUDO_PREFIX=/share/perl6
RAKUDO_MAIN_FILE=/share/perl6/runtime/perl6.moarvm (or perl6-debug.moarvm)

These map more directly to the required locations:
libpaths = $RAKUDO_BASE_LIB
main = $RAKUDO_MAIN_FILE
comp-repos = $RAKUDO_PREFIX, $RAKUDO_PREFIX/site, $RAKUDO_PREFIX/vendor

It makes sense to have a RAKUDO_ instead of a PERL6_ prefix, since these locations are definitely rakudo specific. A different implementation might not even use NQP or need a main file.
Open questions


Is /share/perl6/lib ever used? If not it can be removed.
Is the layout of the comp repos in /share/perl6 guaranteed to be stable (/, /site, /vendor)? If not we need something like "RAKUDO_COMP_REPOS=/share/perl6,/share/perl6/site,/share/perl6/vendor" instead of RAKUDO_PREFIX.
Is there any way to make this simpler? Or are we really stuck with three different configuration options?
Might it be a good idea to derive RAKUDO_MAIN_FILE from RAKUDO_BASE_LIB by simply searching all the folders?
	What Python does
	================


	See `Python-3.7.0/Modules/getpath.c`. Python always determines its library locations at runtime (prefix and exec_prefix).

	It first determines the executable location (robustly, including $PATH and symlink handling), then performs several methods to determine `prefix` and `exec_prefix`, two paths that point to library and shared object folders in the
	installation.

	1. Check if it's run from the build directory (by looking for a marker file). Use a separate path layout matching the build directory in that case.
	2. Look at the $PHTHONHOME environment variable. If it's there, use it.
	3. Search for the installation path relative to the executable location (climb up the folder hierarchy and check every folder encountered for a marker file).
	4. Use the hard coded preprocessor variables.

	Reasoning of the above approach (just guessing):

	1. Build dir path allows running a freshly compiled python during the build phase.
	2. The environment variable to allow overrides.
	3. This is what makes relocatability possible.
	4. This is described as a last fallback that is usually not helpful. But I think it is what most distributions would want to use in a fixed installation scenario.

	I think a better ordering of the above checks would be:

	1. Check for the environment variable.
	2. Use the hard coded variable if it was set in the build.
	3. Search relative to the executable.
	4. Don't implement the build dir check. If that's needed we can simply use the custom runners we already have.

	That way a build is either relocatable (with the hard coded variables unset) or located (with the variables set).

	Skipping the build dir checks prevents some potential funky attack scenarios (place an installation in / or something) and we do not pay for the search. Setting the hard coded variables to empty just skips the search altogether.