Navigation Menu

Skip to content

Instantly share code, notes, and snippets.

@tzi
Last active April 16, 2019 12:35
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save tzi/5bd95576feb6224ebb207bf861e06a3c to your computer and use it in GitHub Desktop.
Save tzi/5bd95576feb6224ebb207bf861e06a3c to your computer and use it in GitHub Desktop.
Check host availability from an HTML page

Check domains

This script check domains availability of links from an HTML file 🎉

# Usage
$ ./check-domains.sh ./bookmarks.html
aslibrary.org is available
#!/bin/bash
if [ $# -gt 0 ];
then
input=$1;
else
echo "Usage: $0 <file.html>";
echo ;
exit 1
fi
# Extract urls from HTML file
urls=$(mech-dump --links "$input")
# Extract domains from urls
domains=$(cut -d"/" -f3 <<<"$urls"| rev | cut -d"." -f-2 | rev | sort | uniq);
# For each domain check availability
for domain in $domains;
do
# Check for a server ping
host "$domain" | grep "NXDOMAIN" >&/dev/null;
if [ $? -eq 1 ]
then
continue
fi
# Check on whois registry
whois "$domain" | grep -E "(No match for|NOT FOUND)" >&/dev/null;
if [ $? -eq 1 ]
then
continue
fi
echo "$domain is available";
done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment