Current and proposed crate publish procedure in diff format

crate-publish.md

Proposed

Notes

• We enforce a 50MB max in nginx
• We should add a configuration entry for the global max size of the metadata (we currently use max tarball size several places)
• A few guidelines I tried to follow:
  • Identify and reject invalid requests as quickly as possible.
  • Minimize the work done while holding a database connection, especially after entering the main transaction.
  • The final main transaction may need to repeat some queries to ensure it doesn't rely on data obtained outside of the transaction.

Verify Headers

• Verify user is authenticated
• Check that Content-Length header exists and doesn't exceed global max tarball + global max metadata + 2 size fields

Verify Request Body

• Check metadata length against the global max metadata size
• Read in metadata
• Read in tarball size, verify tarball size + metadata size + 2 size fields == Content-Length
• Decode metadata
• Check for non-empty: description, license, authors
• Validate URLs if present: homepage, documentation, repository
• Validate license if specified
• Iterate over deps
  • Enforce "no wildcard" constraint on deps
  • Check that not an alternate registry dependency
• Validate deserialization of badges into enum, collect invalid ones
• Read remaining request body
• Verify tarball
• Calculate crate tarball hash (for registry)
• Render README to HTML
• Upload crate and rendered README (if not --dry-run)

With database, outside of main transaction

• Obtain database connection
• Ensure user has a verified email address
• Ensure name is not reserved
• Obtain a list of valid and invalid categories
• Ensure that all deps exist
• If crate exists
  • Check that the new name is identical to the existing name (sans-canonicalization)
  • Verify tarball doesn't exceed the crate specific max
  • Check that the user has publish rights on the crate
• If crate didn't exist
  • Check the rate limit
  • Verify tarball doesn't exceed default max
• Check if the version already exists
• --dry-run check

Start writing within the transaction

• Enter database transaction
• If crate didn't exist then insert and add the user as an owner
• If crate was present, update it (TODO: review what fields on the crate we update under which circumstances. How do we deal with prerleases (#1389) and backports?)
• Insert version (abort if exists) and add authors to the version
• Record README rendered_at for version
• Iterate over deps
  • Handle package renames
  • Insert deps into database
• Update keywords
• Update categories
• Update badges
• Use database to obtain max_version of the crate (for response)
• Iterate over deps to get a vec of git::Dependency
• Upload crate
• Background jobs
  • If readme provided, enqueue rendering and upload as a background job
  • Enqueue index update
• Commit database transaction
• Encode response