Skip to content

Instantly share code, notes, and snippets.

@gabefair

gabefair/atwarrior.md

Forked from catleeball/atwarrior.md
Created April 27, 2023 21:03
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save gabefair/9c16664004e08d495900681cdb51ce9a to your computer and use it in GitHub Desktop.
Save gabefair/9c16664004e08d495900681cdb51ce9a to your computer and use it in GitHub Desktop.
Download ZIP
Tutorial on running an Archive Team Warrior container with Docker Compose
Raw
atwarrior.md

You can run an Archive Team container on your computer! It'll help crawl the web and putting stuff into archive.org, based on the current projects.

You don't have to be super technical to do this, but a little computer skill will help you out.

Walkthrough below the cut:

The official docs are here: https://wiki.archiveteam.org/index.php/ArchiveTeam_Warrior

Below I'll try to explain in a way that's a smidge more accessible if you're not computer wizard.

Note that the example I used below adds a little extra stuff to make sure DNS requests are cached and use a resolver that doesn't filter requests that the main docs don't do.1

Prerequisites

  • Extra bandwidth2
  • A computer3
  • Docker and Docker Compose installed on the computer
  • Able to run commands in your terminal

Setup

  1. Make a file named docker-compose.yml on your computer.
  2. Copy + Paste this configuration into it
  3. Maybe uncomment4 and update fields to your preferences, such as:
  • DOWNLOADER=Your Cool Username for the Leaderboards
  • HTTP_USERNAME and HTTP_PASSWORD to add a basic login page for the tool
  • volumes: ... This will let the container store stuff on your disk for use between container restarts. You need to replace the path before the colon with a local directory. If you don't know what this means, just ignore it. :)

Running

  1. Open a terminal
  2. Move to the directory where you saved docker-compose.yml
  • e.g. cd /home/name/some_folder/
  1. Run sudo docker compose -d bind watchtower at_warrior
  • Some installations you need a hyphen in docker-compose

Usage

  1. Open a browser on the same computer this is running
  2. Go to http://localhost:8010
  3. You should see a website like this running on your machine:

Screenshot of web UI

That's it! There's more settings you can dink with, including which project it should run (it will pick one automatically by default), or just close the page and rest happy knowing you're helping backup dying websites before they go offline.

FAQ

  • Are you affiliated with Archive Team?
    • No! I just run a container like I described above on my computer.
  • Is Archive Team backing up Imgur? I hear it's going down soon!
    • Yes, but the project isn't on the Warrior container yet! There's a page on the wiki here including the IRC channel folks are apparently coordinating the project in.

Footnotes

Footnotes

  1. I did try to submit this to the official wiki page, but it's awaiting approval. ↩

  2. If you find yourself often struggling to stream video, running this on the same network will make it even harder to stream video, since this will do a lot of downloading and uploading. ↩

  3. Ideally, the computer will be one that's always powered on, or at least most of the time. Probably not a laptop. Your laptop will get hot and the battery will die fast. ↩

  4. If the line starts with a #, it's a "comment". For it to be used, you need to remove the # and make sure the spaces still line up. ↩

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment