Skip to content

Instantly share code, notes, and snippets.

@kimsible

kimsible/README.md

Last active Sep 10, 2020
Embed
What would you like to do?
PeerTube admin tools for docker

Prerequisites

  • Domain Name
  • Main Dedicated Server
  • Private FTP Backups Server

General budget: from ~17€ to ~100€ per month

Domain Name

Obviously you need a domain name for 15/20€ per year pointing to the IP of the Main Dedicated server ;-)

Be sure to configure DKIM TXT DNS record after installing PeerTube Stack.

Main dedicated server

Minimum hardware

  1. ~10€ per month for small channels with few and short content
  2. ~45€ per month for channels with average and moderate content and maybe one channel with lots of content
  3. ~100€ per month for several channels with very regular and big content
  • CPU X86_64:

    1. Intel Avoton / Atom with at least 2 cores but slow for transcoding
    2. Intel Core i3, Intel Xeon
    3. Intel Core i5, Intel Xeon
  • RAM:

    1. >=2Go
    2. >=8Go
    3. >=32Go
  • HDD:

    1. <= 500Go
    2. >= 2To
    3. > 4To
  • Bandwith:

    1. >= 150Mbps / Unlimited
    2. >= 250Mbps / Unlimited
    3. >= 500Mbps / Unlimited

Source: https://github.com/Chocobozzz/PeerTube/blob/develop/FAQ.md#should-i-have-a-big-server-to-run-peertube

Operating System

PeerTube Docker Stack

Installing with one shell-command: https://github.com/kimsible/install-peertube/

Private FTP server for daily backup

At least 100Go, generally free and provided with the Main Dedicated Server, used to backup all the Docker data-volume and setup including database with videos files.

PeerTube Backup Script

Creation and usage: As root or sudoer user

  • Install curl
  • Copy peertube-backup content to /usr/bin/peertube-backup
  • Define SERVER_URL=user:password@address.tld according to your needs
  • Make /usr/bin/peertube-backup executable with chmod +x /usr/bin/peertube-backup
  • Proceed backup by running peertube-backup (will overwrite existing daily backup on ftp server)
  • The backup will be avalaible on backup ftp server

PeerTube Daily Backups

Creation and usage: As root or sudoer user

  • Edit crontab with crontab -u root -e
  • Add this line to run it as docker user every day at 5:25am :
25 5  * * * /usr/bin/peertube-backup

PeerTube Restore Script

Creation and usage: As root or sudoer user

  • Install curl, lftp
  • Copy peertube-restore content to /usr/bin/peertube-restore
  • Define SERVER_URL=user:password@address.tld according to your needs
  • Make /usr/bin/peertube-restore executable with chmod +x /usr/bin/peertube-restore
  • Proceed restore by running peertube-restore peertube-backup_YYYY-MM-DD.tar.bz2

Migrate old server to local new server

Creation and usage: As root or sudoer user

scp -r -p username@old-server:/var/peertube /var/peertube

PeerTube CLI Server Tools scripts

Creation: As root or sudoer user

Usage: As non-root user with docker group

  • Copy peertube-cli content to /usr/bin/peertube
  • Make /usr/bin/peertube executable with chmod +x /usr/bin/peertube
  • Run any Server Tool script with peertube SERVER_TOOL

Server Tools List: https://docs.joinpeertube.org/#/maintain-tools?id=server-tools

PeerTube Assets Update - DEPRECATED

This feature is included in PeerTube with client-overrides since v2.3.0: https://github.com/Chocobozzz/PeerTube/releases/tag/v2.3.0

Creation and usage: As root or sudoer user

  • Copy peertube-assets content to /usr/bin/peertube-assets
  • Make /usr/bin/peertube-asssets executable with chmod +x /usr/bin/peertube-assets
  • Copy peertube-assets.service content to /etc/systemd/system/peertube-assets.service
  • Run systemctl daemon-reload and systemctl enable peertube-assets
  • Put your assets in /var/peertube/docker-volume/data/client-overrides/assets with scp -r -p /local-assets-dir username@main-server:/var/peertube/docker-volume/data/assets
  • Run systemctl restart peertube
#!/usr/bin/env sh
# Constants
COMPOSE=/usr/local/bin/docker-compose
# Go to workir
cd /var/peertube
# Create assets volume if non-exists
mkdir -p ./docker-volume/data/client-overrides/assets
if [ ! -f docker-volume/data/client-overrides/assets/images/logo.svg ] && [ ! -f docker-volume/data/client-overrides/assets/images/favicon.png ] && [ ! -f docker-volume/data/client-overrides/assets/images/default-playlist.jpg ]; then
echo 'No assets to update'
exit 1
fi
# Do not update until server is up
echo -n "Wait until PeerTube server is up ... "
sleep 50s &
while [ -z "`$COMPOSE logs --tail=2 peertube | grep -o 'Server listening on'`" ]; do
# Break if any stack errors occur
# Displays errors and exit
stack_errors=`$COMPOSE logs --tail=40 peertube | grep -i 'error'`
if [ ! -z "$stack_errors" ]; then
echo $stack_errors
exit 1
fi
# Break after 50s / until pid of "sleep 50s" is destroyed
# Display logs and exit
if [ -z "`ps -ef | grep $! | grep -o -E 'sleep 50s'`" ]; then
$COMPOSE logs --tail=40 peertube
exit 1
fi
done
echo "done"
# logo.svg
if [ -f docker-volume/data/client-overrides/assets/images/logo.svg ]; then
echo -n 'Update images/logo.svg ... '
docker cp ./docker-volume/assets/images/logo.svg $(docker ps -q --filter "name=peertube_peertube_1"):/app/client/dist/assets/images/logo.svg
echo 'done'
fi
# favicon.png
if [ -f docker-volume/data/client-overrides/assets/images/favicon.png ]; then
echo -n 'Update images/favicon.png ... '
docker cp ./docker-volume/data/client-overrides/assets/images/favicon.png $(docker ps -q --filter "name=peertube_peertube_1"):/app/client/dist/assets/images/favicon.png
echo 'done'
fi
# default-playlist.jpg
if [ -f docker-volume/data/client-overrides/assets/images/default-playlist.jpg ]; then
echo -n 'Update images/default-playlist.jpg ... '
docker cp ./docker-volume/data/client-overrides/assets/images/default-playlist.jpg $(docker ps -q --filter "name=peertube_peertube_1"):/app/client/dist/assets/images/default-playlist.jpg
echo 'done'
fi
# icons/icon-36x36.png
if [ -f docker-volume/data/client-overrides/assets/images/icons/icon-36x36.png ]; then
echo -n 'Update images/icons/icon-36x36.png ... '
docker cp ./docker-volume/data/client-overrides/assets/images/icons/icon-36x36.png $(docker ps -q --filter "name=peertube_peertube_1"):/app/client/dist/assets/images/icons/icon-36x36.png
echo 'done'
fi
# icons/icon-48x48.png
if [ -f docker-volume/data/client-overrides/assets/images/icons/icon-48x48.png ]; then
echo -n 'Update images/icons/icon-48x48.png ... '
docker cp ./docker-volume/data/client-overrides/assets/images/icons/icon-48x48.png $(docker ps -q --filter "name=peertube_peertube_1"):/app/client/dist/assets/images/icons/icon-48x48.png
echo 'done'
fi
# icons/icon-72x72.png
if [ -f docker-volume/data/client-overrides/assets/images/icons/icon-72x72.png ]; then
echo -n 'Update images/icons/icon-72x72.png ... '
docker cp ./docker-volume/data/client-overrides/assets/images/icons/icon-72x72.png $(docker ps -q --filter "name=peertube_peertube_1"):/app/client/dist/assets/images/icons/icon-72x72.png
echo 'done'
fi
# icons/icon-96x96.png
if [ -f docker-volume/data/client-overrides/assets/images/icons/icon-96x96.png ]; then
echo -n 'Update images/icons/icon-96x96.png ... '
docker cp ./docker-volume/data/client-overrides/assets/images/icons/icon-96x96.png $(docker ps -q --filter "name=peertube_peertube_1"):/app/client/dist/assets/images/icons/icon-96x96.png
echo 'done'
fi
# icons/icon-144x144.png
if [ -f docker-volume/data/client-overrides/assets/images/icons/icon-144x144.png ]; then
echo -n 'Update images/icons/icon-144x144.png ... '
docker cp ./docker-volume/data/client-overrides/assets/images/icons/icon-144x144.png $(docker ps -q --filter "name=peertube_peertube_1"):/app/client/dist/assets/images/icons/icon-144x144.png
echo 'done'
fi
# icons/icon-192x192.png
if [ -f docker-volume/data/client-overrides/assets/images/icons/icon-192x192.png ]; then
echo -n 'Update images/icons/icon-192x192.png ... '
docker cp ./docker-volume/data/client-overrides/assets/images/icons/icon-192x192.png $(docker ps -q --filter "name=peertube_peertube_1"):/app/client/dist/assets/images/icons/icon-192x192.png
echo 'done'
fi
# icons/icon-512x512.png
if [ -f docker-volume/data/client-overrides/assets/images/icons/icon-512x512.png ]; then
echo -n 'Update images/icons/icon-512x512.png ... '
docker cp ./docker-volume/data/client-overrides/assets/images/icons/icon-512x512.png $(docker ps -q --filter "name=peertube_peertube_1"):/app/client/dist/assets/images/icons/icon-512x512.png
echo 'done'
fi
[Unit]
Description=PeerTube Assets Update Script
Requires=peertube.service
After=peertube.service
PartOf=peertube.service
[Service]
User=docker
Group=docker
WorkingDirectory=/var/peertube
ExecStart=/usr/bin/peertube-assets
[Install]
WantedBy=peertube.service
#!/usr/bin/env sh
#########################################################
# DEEPENDENCIES #
# curl #
#########################################################
#########################################################
# ENVIRONMENT VARIABLES #
SERVER_URL=user:password@address.tld
SERVER_SSL=false
#########################################################
# root
uid=`id -u`
if [ "$uid" -ne 0 ]; then
echo "ERROR: this script must be run as root or as a sudoer user with sudo"
exit 1
fi
# Constants
COMPOSE=/usr/local/bin/docker-compose
DB_TAR=db.tar
CURRENT_DATE=$(date '+%Y-%m-%d_%H-%M-%S')
TRANSCODED_TAR=peertube-backup_$CURRENT_DATE-transcoded.tar
BACKUP_BZIP=peertube-backup_$CURRENT_DATE.tar.bz2
if [ "$SERVER_SSL" = true ]; then
SSL_OPTION⁼'--ssl '
else
SSL_OPTION=''
fi
# Go to workir
cd /var/peertube
# Get postgres user
POSTGRES_USER="`grep -E -o "^POSTGRES_USER=(.+)" .env | sed -E "s/POSTGRES_USER=//g"`"
# Get postgres db
POSTGRES_DB="`grep -E -o "^POSTGRES_DB=(.+)" .env | sed -E "s/POSTGRES_DB=//g"`"
# Get postgres service name
PEERTUBE_DB_HOSTNAME="`grep -E -o "^PEERTUBE_DB_HOSTNAME=(.+)" .env | sed -E "s/PEERTUBE_DB_HOSTNAME=//g"`"
# Do not backup if PostgreSQL container is down
if [ -z "$($COMPOSE ps -q $PEERTUBE_DB_HOSTNAME)" ]; then
echo -n "ERROR: PostgreSQL docker container is down, can't backup"
exit 1
fi
# Get videoFile table size in PostgreSQL database
echo -n "Get number of rows of videoFile table in PostgreSQL database ... "
TRANSCODED_COUNT=`$COMPOSE exec -T $PEERTUBE_DB_HOSTNAME psql -t -U $POSTGRES_USER $POSTGRES_DB -c 'SELECT count(*) FROM "videoFile"'`
echo "done"
# Get videoFile table last id in PostgreSQL database
echo -n "Get last id of videoFile table in PostgreSQL database ... "
TRANSCODED_LAST=`$COMPOSE exec -T $PEERTUBE_DB_HOSTNAME psql -t -U $POSTGRES_USER $POSTGRES_DB -c 'SELECT max(id) FROM "videoFile"'`
echo "done"
# Dump PostgreSQL database
echo -n "Dump existing PostgreSQL database ... "
$COMPOSE exec -T $PEERTUBE_DB_HOSTNAME pg_dump -U $POSTGRES_USER -Ft $POSTGRES_DB > docker-volume/$DB_TAR
echo "done"
# Archive and Compress DB Dump, Docker volume and Compose files
# Do not include mounted db though db.tar has been created
# Do not include transcoded video files in /data/videos and /data/streaming-playlists
# Send the compressed file to ftp
echo "Archive, compress and send to ftp existing database dump, Docker volume and Compose files ... "
tar --exclude=docker-volume/db --exclude=docker-volume/data/videos --exclude=docker-volume/data/streaming-playlists -cjf /tmp/peertube-backup.tar.bz2 docker-volume .env docker-compose.yml
curl $SSL_OPTION-aT /tmp/peertube-backup.tar.bz2 ftp://$SERVER_URL/$BACKUP_BZIP
rm -f /tmp/peertube-backup.tar.bz2
echo "done"
# Clean db.tar
echo -n "Clean archived database dump ... "
rm -f docker-volume/$DB_TAR
echo "done"
# Init markers if needed
if [ ! -f docker-volume/data/.videocount ] || [ ! -f docker-volume/data/.videolast ]; then
echo -n "Init current number of transcoded video files and last index markers... "
echo 0 > docker-volume/data/.videocount
echo 0 > docker-volume/data/.videolast
echo "done"
fi
# Send transcoded videos files to ftp if needed
if [ "$TRANSCODED_COUNT" -ne "`cat docker-volume/data/.videocount`" ] || [ "$TRANSCODED_LAST" -ne "`cat docker-volume/data/.videolast`" ]; then
echo "Archive and send transcoded video files ... "
tar -cf /tmp/peertube-backup-transcoded.tar docker-volume/data/videos docker-volume/data/streaming-playlists
curl $SSL_OPTION-aT /tmp/peertube-backup-transcoded.tar ftp://$SERVER_URL/$TRANSCODED_TAR
rm -f /tmp/peertube-backup-transcoded.tar
echo "done"
fi
# Set current number of transcoded videos files and the last index in db to backup only if needed
echo -n "Set current number of transcoded video files and last index ... "
echo $TRANSCODED_COUNT > docker-volume/data/.videocount
echo $TRANSCODED_LAST > docker-volume/data/.videolast
echo "done"
#!/usr/bin/env sh
# Constants
COMPOSE=/usr/local/bin/docker-compose
SERVER_TOOLS="
parse-log
create-transcoding-job
create-import-video-file-job
prune-storage
optimize-old-videos
update-host
reset-password
plugin:install
plugin:uninstall
purge-old-account
psql
"
# Limit command to Server Tools List
if [ -z $1 ] || [ -z "`echo $SERVER_TOOLS | grep -o -w $1`" ]; then
echo "\nUsage: peertube SERVER_TOOL\n"
echo "Server Tools:\n$SERVER_TOOLS"
exit 1
fi
# Go to workir
cd /var/peertube
# Get postgres user
POSTGRES_USER="`grep -E -o "^POSTGRES_USER=(.+)" .env | sed -E "s/POSTGRES_USER=//g"`"
# Get postgres db
POSTGRES_DB="`grep -E -o "^POSTGRES_DB=(.+)" .env | sed -E "s/POSTGRES_DB=//g"`"
# Get postgres service name
PEERTUBE_DB_HOSTNAME="`grep -E -o "^PEERTUBE_DB_HOSTNAME=(.+)" .env | sed -E "s/PEERTUBE_DB_HOSTNAME=//g"`"
if [ "$1" = "purge-old-account" ]; then
username=$2
if [ -z "$username" ]; then
echo "\nUsage: peertube purge-old-account USERNAME\n"
exit 1
fi
count=`$COMPOSE exec -T $PEERTUBE_DB_HOSTNAME psql -t -U $POSTGRES_USER $POSTGRES_DB -c "SELECT count(*) FROM account WHERE name = '$username';" | xargs`
if [ "$count" -gt 0 ]; then
echo "ERROR: can't purge an existing account"
exit 1
fi
# Run psql delete command
channelname=$username"_channel"
$COMPOSE exec -T $PEERTUBE_DB_HOSTNAME psql -e -U $POSTGRES_USER $POSTGRES_DB -c "DELETE FROM actor WHERE \"preferredUsername\" = '$username';"
$COMPOSE exec -T $PEERTUBE_DB_HOSTNAME psql -e -U $POSTGRES_USER $POSTGRES_DB -c "DELETE FROM actor WHERE \"preferredUsername\" = '$channelname';"
elif [ "$1" = "psql" ]; then
# Run psql command
$COMPOSE exec -T $PEERTUBE_DB_HOSTNAME psql -U $POSTGRES_USER $POSTGRES_DB -c "$2"
else
# Run ServerTools command
$COMPOSE exec -u peertube -e NODE_CONFIG_DIR=/config -e NODE_ENV=production peertube npm run $@
fi
#!/usr/bin/env sh
#########################################################
# DEEPENDENCIES #
# lftp #
# curl #
#########################################################
#########################################################
# ENVIRONMENT VARIABLES #
SERVER_CREDENTIALS=user:password
SERVER_ADDRESS=address.tld
SERVER_SSL=false
#########################################################
# root
uid=`id -u`
if [ "$uid" -ne 0 ]; then
echo "ERROR: this script must be run as root or as a sudoer user with sudo"
exit 1
fi
# Constants
BACKUP=$1
COMPOSE=/usr/local/bin/docker-compose
if [ "$SERVER_SSL" = true ]; then
SCHEME⁼'ftps'
SSL_CMD=''
SSL_OPTION='--ssl '
else
SCHEME='ftp'
SSL_CMD='set ssl:verify-certificate false;'
SSL_OPTION=''
fi
# Backup name argument must exist and must match ^peertube-backup_[0-9]{4}-[0-9]{2}-[0-9]{2}(-transcoded)?\.tar(\.bz2)?$
# Otherwise, display usage and all backups from ftp
match=`echo $BACKUP | grep -E -o "^peertube-backup_[0-9]{4}-[0-9]{2}-[0-9]{2}(-transcoded)?\.tar(\.bz2)?$"`
if [ -z $BACKUP ] || [ -z $match ]; then
echo "\nUsage: peertube-restore peertube-backup_YYYY-MM-DD(-transcoded).tar(.bz2)\n"
lftp $SCHEME://$SERVER_CREDENTIALS@$SERVER_ADDRESS -e "$SSL_CMD ls; quit"
exit 1
fi
# Get tar options
bzip=`echo $BACKUP | grep -E -o "bz2$"`
if [ -z $bzip ]; then
tar_option='xf'
else
tar_option='xjf'
fi
# Stop service to not conflict restore
echo -n "Stop systemd service to not conflict restore ... "
systemctl stop peertube
echo "done"
# Get backup to restore from ftp into buffer and extract it
echo "Get $BACKUP from ftp backup server and extract it to /var/peertube ... "
curl -L $SSL_OPTION-u $SERVER_CREDENTIALS ftp://$SERVER_ADDRESS/$BACKUP -o /tmp/$BACKUP
tar -$tar_option /tmp/$BACKUP -C /var/peertube
rm -f /tmp/$BACKUP
echo "done"
# Go to workdir
cd /var/peertube
# Restore db.tar if exists
if [ -f ./docker-volume/db.tar ] && [ -f .env ] && [ -f .docker-compose.yml ]; then
# Get postgres user
POSTGRES_USER="`grep -E -o "^POSTGRES_USER=(.+)" .env | sed -E "s/POSTGRES_USER=//g"`"
# Get postgres db
POSTGRES_DB="`grep -E -o "^POSTGRES_DB=(.+)" .env | sed -E "s/POSTGRES_DB=//g"`"
# Get postgres service name
PEERTUBE_DB_HOSTNAME="`grep -E -o "^PEERTUBE_DB_HOSTNAME=(.+)" .env | sed -E "s/PEERTUBE_DB_HOSTNAME=//g"`"
# Remove existing mounted db files
echo -n "Remove existing mounted /var/docker-volume/db ... "
rm -rf ./docker-volume/db
sleep 1s
echo "done"
# Up postgres service
$COMPOSE up -d $PEERTUBE_DB_HOSTNAME
echo -n "Wait until PosgreSQL database is up ... "
sleep 10s &
while [ -z "`$COMPOSE logs --tail=2 $PEERTUBE_DB_HOSTNAME | grep -o 'database system is ready to accept connections'`" ]; do
# Break if any database errors occur
# Displays errors and exit
db_errors=`$COMPOSE logs --tail=40 $PEERTUBE_DB_HOSTNAME | grep -i 'error'`
if [ ! -z "$db_errors" ]; then
echo $db_errors
exit 1
fi
# Break after 10s / until pid of "sleep 10s" is destroyed
# Display logs and exit
if [ -z "`ps -ef | grep $! | grep -o -E 'sleep 10s'`" ]; then
$COMPOSE logs --tail=40 $PEERTUBE_DB_HOSTNAME
exit 1
fi
done
echo "done"
# Run restore command
echo -n "Restore existing dump /var/peertube/docker-volume/db.tar ... "
$COMPOSE exec -T $PEERTUBE_DB_HOSTNAME pg_restore -U $POSTGRES_USER -d $POSTGRES_DB < ./docker-volume/db.tar
echo "done"
# Clean db.tar
echo -n "Clean existing dump /var/peertube/docker-volume/db.tar ... "
rm -f ./docker-volume/db.tar
echo "done"
fi
# Re-start service
echo -n "Re-start stopped systemd service ... "
systemctl start peertube
echo "done"
@rigelk

This comment has been minimized.

Copy link

@rigelk rigelk commented Jun 5, 2020

You might want to limit your SQL query of videoFiles to local videos.

@kimsible

This comment has been minimized.

Copy link
Owner Author

@kimsible kimsible commented Jun 5, 2020

You might want to limit your SQL query of videoFiles to local videos.

Thanks, you're right, didn't think about it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.