Skip to content

Instantly share code, notes, and snippets.

### Keybase proof
I hereby claim:
* I am rooneymcnibnug on github.
* I am atomsk (https://keybase.io/atomsk) on keybase.
* I have a public key ASDG4cphU7n4RO1YDd5nA6G78_Ah1X-hZ7WjGA9o353G6Qo
To claim this, I am signing this object:
@RooneyMcNibNug
RooneyMcNibNug / SA_public_thread_scrape.py
Last active December 6, 2023 16:57
For when you want to scrape a publicly available thread (including images) on forums.somethingawful.com
### Python script using bs4 to scrape a Something Awful forums thread. ###
#
# ! THIS WILL ONLY WORK ON THREADS THAT ARE PUBLICLY ACCESSIBLE, IE ONE'S YOU DON'T NEED TO LOG IN TO SEE !
# (I didn't want to deal with auth stuff..)
#
# Make sure you replace the thread_url variable with the link you want to scrape, same as the example in the code here.
#
# This will dump to an HTML file with decent formatting (a bit too wall-of-text atm, but I have too many other things to tend to in life).
#
# If you want to have better file naming including something like the ThreadID, do this:
# In case you need your raspberry pi model:
cat /sys/firmware/devicetree/base/model
# See versions of raspbian/debian currently supported by pi-hole:
dig +short -t txt versions.pi-hole.net @ns1.pi-hole.nets.pi-hole.net @ns1.pi-hole.net
"Raspbian=10,11,12 Ubuntu=20,22,23,24 Debian=10,11,12 Fedora=39,40 CentOS=9"
# update packages on current OS version:
sudo apt-get update && sudo apt-get upgrade