Running ArchiveTeam Warrior on GCP

README.md

Running ArchiveTeam Warrior on GCP

Guide by riley (efnet, @bad.pet)

Create a GCP project

If necessary. https://console.cloud.google.com/projectcreate
Then select the project you want to use

Create a new firewall rule

https://console.cloud.google.com/networking/firewalls/add
Defaults, except:
Name = archiveteam-warrior-http
Target tags = archiveteam-warrior
Source IP ranges = 0.0.0.0/0 (or your own IP, /32)
[x] tcp: 8001-8006

Create a new instance template

https://console.cloud.google.com/compute/instanceTemplates/add
name: archiveteam-warrior
type: micro
disk: 10gb debian 9
Expand "Management, security, disks, networking, sole tenancy" section
Security -> SSH Keys -> paste in your key
Networking -> Network service tier, if you like
Networking -> Network tags = archiveteam-warrior
Management -> Startup script (edit the variables!):

#!/bin/bash

DOWNLOADER="awoobis-unconfigured" # your handle for the scoreboard
USERNAME="YOUR HTTP USERNAME HERE"
PASSWORD="PICK A PASSWORD"
PROJECT="auto"
INSTANCE_COUNT="1"
CONCURRENCY="6" # simultaneous items to run PER INSTANCE
SWAPSIZE="256M"

# A bit of swap for comfort
fallocate -l "$SWAPSIZE" /swap
chmod 600 /swap
mkswap /swap
echo '/swap swap swap defaults 0 0' >>/etc/fstab
swapon /swap

# Configure docker debian repo and install docker-ce
apt-get update
apt-get install -y apt-transport-https ca-certificates \
    curl gnupg2 software-properties-common
curl -fsSL https://download.docker.com/linux/debian/gpg | sudo apt-key add -
add-apt-repository "deb [arch=amd64] \
    https://download.docker.com/linux/debian \
    $(lsb_release -cs) stable"
apt-get update
apt-get install -y docker-ce

# Start warrior instances
for x in `seq 1 $INSTANCE_COUNT` ; do
    port=$((8000 + x))
    docker run --detach \
        --env DOWNLOADER="$DOWNLOADER" \
        --env SELECTED_PROJECT="$PROJECT" \
        --env CONCURRENT_ITEMS="$CONCURRENCY" \
        --env HTTP_USERNAME="$USERNAME" \
        --env HTTP_PASSWORD="$PASSWORD" \
        --publish $port:8001 \
        --restart no \
        archiveteam/warrior-dockerfile
done

Then start deploying!

https://console.cloud.google.com/compute/instancesAdd?templateName=archiveteam-warrior
It'll take several (up to ~9) minutes for setup to finish, but eventually you should be able to go to http://$vm_ip:8001 etc to see the status panel. To watch what it's doing, just SSH to the VM and use ps faux/htop/whatever.

To stop

For a nice shutdown, figure out where the hell the STOP file goes, and run this on each VM:

stopfile="/SOMEWHERE/"
sudo docker ps -q | while read cid ; do sudo docker exec "$cid" /usr/bin/touch $stopfile ; done

Or just go to each web panel and hit "shut down" then wait for it to stop. When you're ready, kill them from https://console.cloud.google.com/compute/instances