Mirror Guidelines

mirrors.yml

# Solus Mirror Network

To help distribute software to our users, Solus relies on a network of volunteer mirror servers to host our respoitories.  Mirrors can be accessed directly by end-users, however by default Solus downloads from our official Tier 1 mirror, which will then redirect the client to a Tier 2 in our network for that user. This assists with load-balancing our clients across our mirror network based on various metrics such as geographic location, bandwidth, and sync status of each server, and also ensures end-users do not suffer from out-of-date mirrors.

This document is aimed at system administrators who wish to contribute a server to this network.

To contact the Solus mirror admin team, send a mail to mirrors@solus-project.com.

## Tier 2 Mirrors

Official Solus mirrors are considered *Tier 1* mirrors. Public servers joined to the official pool are *Tier 2* mirrors. These sync from the Tier 1 mirrors, and may or may not be owned and operated by the Solus Project; most are donated by individuals and organizations wishing to contribute bandwidth to the project.

### Requirements

To become a Tier 2 Mirror, you must be able to meet the following criteria:
  - Accessible via **HTTPS**
    - *Servers only accessible via HTTP may become Tier 3 mirrors, but will not be accepted to the official pool.*
  - Accessible via **rsync**
  - Disk-space >= **80 GB**
  - Upload Bandwidth >= **10Mbit/s**
  - Sync off the Tier 1 Mirror; not more often than every hour, but at least twice a day
 
We prefer if you can host the entire upstream repositories, including ISO images.

The Tier 1 mirror must be able to regularly scan Tier 2 mirrors to verify their copy of the tree - this ensures clients are never redirected to an out-of-date mirror. This scan is performed via rsync so we require all Tier 2 mirrors to provide rsync preferably to the public for other new Tier 2 mirrors to perform an initial sync. If required you can restrict access only to the Tier 1 mirror for scanning purposes.

### Joining the Pool

Once you have configured your server and performed an initial sync from another Tier 2 mirror, please follow these steps to join the official pool. We recommend using [this script](https://gist.github.com/fukawi2/adddfca36070df8554ab9c997d602587/raw/ddc1aa84cc1557627bc318d8500eeb0c4cda4abe/sync-solus-mirror.sh), however feel free to utilize your own.

1. Send an email to foobar@example.com containing the following information:

> HTTPS URI: https://your.domain.com/solus/ 
> RSYNC URI: rsync://your.domain.com/solus/
> Sync Schedule: Every x hrs
> Bandwidth: 
> Geographic Location:
> Administrative Email: (will be kept private)

2. Join the solus-mirrors mailing list.
3. Once joined to the pool, you will receive details of how to access the Tier 1 mirror for syncing.

## Tier 3 Mirrors

Tier 3 mirrors are unofficial hosts not part of the official network.  End-users may choose to use a Tier 3 mirror, but they will not be referred to a Tier 3 mirror from the offical redirector.

## Technical Details

[MirrorBrain](http://mirrorbrain.org) is used to redirect end-users to mirrors within the network. MirrorBrain uses [various mirror selection criteria](http://mirrorbrain.org/features/) when redirecting a client. Importantly, mirrors will receive traffic proportionate to their bandwidth. For example, a server with 1gbit upload may receive approximately 10 times the traffic of a server with 100mbit upload.

sync-solus-mirror.sh

#!/bin/bash

# This script based on: https://git.server-speed.net/users/flo/bin/tree/

set -eu

# Directory where the repo is stored locally. Example: /srv/repo
target=''

# The source URL of the mirror to sync from.
source_url=''

# Lockfile path
lock="/tmp/syncrepo.lck"

# If you want to limit the bandwidth used by rsync set this.
bwlimit=0

function usage() {
  echo "Usage: $0 -t target_dir -s source_url [-b bwlimit] [-P rsync_password]"
}

function rsync_cmd() {
  local -a cmd=(rsync -rtlH --safe-links --delete-delay --prune-empty-dirs \
                "--timeout=600" "--contimeout=60" --perms --delay-updates )

  # if we're running without a terminal (eg, under cron) then make
  # ourself run quietly, otherwise give the user some detail
  if stty &>/dev/null; then
    cmd+=(--human-readable --verbose --progress)
  else
    cmd+=(--quiet --no-motd)
  fi

  if ((bwlimit>0)); then
    cmd+=("--bwlimit=$bwlimit")
  fi

  "${cmd[@]}" "$@"
}

function main() {
  while getopts "hs:t:b:P:" opt; do
    case $opt in
    t)
      target="$OPTARG"
      ;;
    s)
      source_url="$OPTARG"
      ;;
    P)
      export RSYNC_PASSWORD="$OPTARG"
      ;;
    b)
      bwlimit="$OPTARG"
      ;;
    *)
      usage
      exit 0
      ;;
    *)
      usage >&2
      exit 1
      ;;
    esac
  done

  # validate user input
  [[ -z "$target" ]]      && { usage >&2; exit 1; }
  [[ -z "$source_url" ]]  && { usage >&2; exit 1; }

  # take our lockfile
  exec 9>"${lock}"
  flock -n 9 || exit

  # do the sync from the upstream host
  rsync_cmd "${source_url}"/ "${target}"

  # update lastsync file
  date --rfc-3339=seconds --utc > ${target}/lastsync
  date --rfc-3339=seconds       >> ${target}/lastsync

  return 0
}

main "$@"