Skip to content

Instantly share code, notes, and snippets.

@dagolden
Last active August 29, 2015 13:57
Show Gist options
  • Save dagolden/9531860 to your computer and use it in GitHub Desktop.
Save dagolden/9531860 to your computer and use it in GitHub Desktop.
This document describes the process by which PAUSE analyzes a Perl 5
distribution tarball after upload. This does *NOT* necessarily apply to
perl core uploads.
# Definitions
* 'uploader' → PAUSE ID of a person uploading a file to PAUSE
* 'tarball' → filename of an archive containing a CPAN distribution
(whether tar or otherwise)
* 'distribution' → a collection of files intended to be installed into a
Perl library path; a distribution contains zero or more modules
* 'module' → a .pm file containing zero or more packages
* 'package' → a Perl 5 namespace
* 'distfile' → the "$uploader/$tarball" path
* 'distname' → the leading name part of a tarball, roughtly up to last word
after the last hypen, but before the first period (see CPAN::DistnameInfo)
* 'distpkg' → the distname converted to a package by s{-}{::}g (or another
heuristic algorithm)
* 'distdir' → the unpacked archive
* 'metadata' → a META spec data structure from a META.json or META.yml file
in the distdir
* 'has permissions' → is authorized first-come, primary or co-maint for a
package in the perms table (checked case insensitively)
* 'in the perms table' → exists in permissions, but checked
case-insensitively
# Current behavior
1. If the tarball matches heuristics to identify a dev release (has "_" in
a certain place or has "-TRIAL" before the suffix), then stop processing.
2. Find all packages & versions inside modules in the distdir: if
'provides' metadata exists, use it; otherwise, scan directories using
various heuristics and filter out packages based on 'no_index' metadata, if
available.
3. Start a transaction
- For any packages found that are not in the perms table, add them: if an
'x_authority' field is in metadata, the ID in that field gets first-come
and the uploader gets co-maint, otherwise, the uploader gets first-come.
- Index packages found when the following conditions are true:
* uploader has permissions for the package
* package is not indexed or is indexed using the same case
* package has a non-decreasing version number
- Log indexing successes and failures
- Commit transaction
4. Send an email to the uploader with the results of indexing
# Proposed future behavior
1. If the distpkg exists in the perms table and the uploader doesn't have
permissions for the distpkg, then stop processing.
2. Find all packages & versions inside modules in the distdir. If 'provides'
metadata exists, use it; otherwise, scan directories using various
heuristics and filter out packages based on 'no_index' metadata, if
available.
3. Start a transaction:
- For any packages found that are not in the perms table, add them: if an
'x_authority' field is in metadata, the ID in that field gets first-come
and the uploader gets co-maint, otherwise, the uploader gets first-come.
- Log any permission assignments
- If uploader still doesn't have permissions for the distpkg, rollback and
stop
- If the tarball matches heuristics to identify a dev release (has "_" in a
certain place or has "-TRIAL" before the suffix), then goto Commit
- The list of packages to index includes all packages found, less any
packages marked 'x_private' in 'provides' metadata, if it exists.
- Index each package in the list of packages to index when the following
conditions are true:
* uploader has permissions for the package
* package is not indexed or is indexed using the same case
* package has a non-decreasing version number
- Log indexing successes and failures
- Commit transaction
4. Send an email to the uploader with the results of permissions assignment
and indexing; if processing was stopped for any reason, the reason for the
stop is included in the email. If nothing was logged or rolled back, then
do not send an email.
@dagolden
Copy link
Author

We should consider whether to identify Perl 6 distributions via META.info within the tarball and bail out.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment