Scrape a Twitter Profile for Bio

gistfile1.txt

#! /bin/bash

# Based on https://gist.github.com/scoates/5935435

for u in $*; do
    echo $u
    curl -sL http://twitter.com/$u > /tmp/tw.html
    cat /tmp/tw.html | grep '<title' | perl -pe 's/.*?>//;s/ \(.*//'
    cat /tmp/tw.html | grep ProfileAvatar-image | head -n1 | perl -p -e's/.*?http/http/;s/".*//'
    cat /tmp/tw.html | grep ProfileHeaderCard-bio -a2 | tail -n1 | perl -pe 's/.*?>(.*)<\/p>/\1/;s/<.*?>//g'
    echo
    rm /tmp/tw.html
done

To use this, create a new file called twitter-scrape (or any name of your choosing), make it executable, then:

./twitter-scrape shiflett

You can also pass it multiple usernames:

./twitter-scrape shiflett coates

Or, if you prefer, a file full of usernames:

./twitter-scrape `cat usernames.txt`