foo.p6

use v6.c;

use Pod::Load;

sub recursive-dir($dir) is export {
    my @todo = $dir;
    gather while @todo {
        my $d = @todo.shift;
        for dir($d) -> $f {
            if $f.f {
                take $f;
            }
            else {
                @todo.append: $f.path;
            }
        }
    }
}

sub read-dir($dir) {
    recursive-dir("doc/$dir/")
    .grep({.path ~~ / '.pod6' $/})
    .map({
            .path.subst("doc/$dir/", '')
            .subst(rx{\.pod6$},  '')
            .subst(:g,    '/',  '::')
            => $_
        });

}

sub MAIN {
    my @files = [read-dir($_).Slip for <Type Programs Native Language>][0..100];

    # secuencial
    my $start = now;
    for @files {
        load($_.value);
    }   
    my $end = now;
    my $elapsed = ($end - $start);
    say "Secuencial => $elapsed";

    # parallel
    $start = now;
    my @ts;
    for ^10 {
        @ts.push: start {
            loop {
                last if @files.elems == 0;
                my $file = @files.shift;
                load($file.value);
                CATCH {
                    default {
                        .note;
                    }
                }
            }
        }
    }
    await Promise.allof(@ts);
    $end = now;
    $elapsed = ($end - $start );
    say "Parallel => $elapsed";

}

The problem I see here is that you're accessing shared memory (@files) from many threads. That might cause problems. @files should instead be a channel from where all threads read... Also, you're using Pod::Load anyway, which is supposed not to work in parallel. Does it work?

The problem I see here is that you're accessing shared memory (@files) from many threads

Yes, exactly, you'd need to have a lock around @files. I did not, because I was in a great hurry with this example and to show the approach, this race with @files is not going to hurt a lot while we had an issue with load originally. It goes without saying that if you'll go with this code (which I cannot recommend), it would need a proper locking around @files usage.

Does it work?

It technicaly works, but still sometimes throw Decoder may not... exception, which is caught, and that is a very bad sign. I just showed an another approach compared to race.map that was proposed initially, it still encounters the error, but is likely to "deceive" rakudo that the usage is not concurrent, which it is not, so please, do not use this, this is simply dangerous and will hurt someone's head in the future. My bet in this situation would be to:

1)Are you really sure that load is the most trouble and it has to be parallelized at any price? Maybe there are other gains to find?
2)If yes, then you'd need to talk this over with moarvm dev who is qualified enough to suggest what can and can't be done. Maybe Decoder can be made thread-safe with a couple of locks, maybe there might be some alternative, etc.

Even if Pod::Load works, which it does, there's still the problem of the low-level filesystem routines, and out-of-Rakudo access to filesystem itself. It's probably never a (very) good idea to parallelize that, because it's going to be sequential no matter what. Even if you eliminate a bit of overhead, it's going to come back at you with a vengeance when you go to the OS to do stuff.
Baseline is: disk drives do not have many cores.