Skip to content

Instantly share code, notes, and snippets.

@haccer
Created April 18, 2018 07:26
Show Gist options
  • Save haccer/075d17de37cc78b48db4c46d99b57fd1 to your computer and use it in GitHub Desktop.
Save haccer/075d17de37cc78b48db4c46d99b57fd1 to your computer and use it in GitHub Desktop.
Gather domains in the Alexa top 1 million
#!/bin/bash
# This script will fetch the Alexa top 1 million domains and prepare them how I like it.
# Check if unzip is installed
[ -f /usr/bin/unzip ] || sudo apt-get install unzip
# Get Alexa Top 1 mil
wget -q --show-progress http://s3.amazonaws.com/alexa-static/top-1m.csv.zip
# Unzip
unzip top-1m.csv.zip
# Parse
cat top-1m.csv | cut -d, -f2 > alexa-top-1m.txt
# Finished
echo "[+] Saved to alexa-top-1m.txt"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment