Skip to content

Instantly share code, notes, and snippets.

@pbinkley

pbinkley/watch.py

Created April 1, 2022 14:20
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save pbinkley/2ab5355bab02de78a668ed907127531b to your computer and use it in GitHub Desktop.
Save pbinkley/2ab5355bab02de78a668ed907127531b to your computer and use it in GitHub Desktop.
Download ZIP
Script to tail list of files downloaded by browsertrix
Raw
watch.py
import tailer
import json
import urllib.parse
for line in tailer.follow(open("crawls/collections/library-sumdu-edu-ua/pages/pages.jsonl")):
data = json.loads(line)
url = data['url']
print(urllib.parse.unquote(url))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment