Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Get Bandcamp tags from autocomplete suggestions
import requests
from json import dumps
from csv import writer
import linecache
import sys
csv_file = open('data.csv', 'a+')
csv_writer = writer(csv_file)
current_index = int(linecache.getline('current_index.txt',1))
trigram = linecache.getline('tri.txt',current_index).strip()
if trigram == '':
sys.exit()
url = "https://bandcamp.com/api/fansignup/1/search_tag"
headers = {
'Content-Type': 'text/plain'
}
request = { 'search_term': trigram, 'count': 2 }
payload = dumps(request)
response = requests.request("POST", url, headers = headers, data = payload)
response_json = response.json()
for tag in response_json['matching_tags']:
tag_line = [tag['tag_norm_name'], tag['count'], tag['tag_name']]
csv_writer.writerow(tag_line)
with open('current_index.txt', 'w') as index_file:
index_file.write(str(current_index + 1))
index_file.close()
@joachimesque

This comment has been minimized.

Copy link
Owner Author

@joachimesque joachimesque commented Nov 14, 2020

I run it every 3 seconds with the help of bash and cron

* * * * * ~/run_python_scraper.sh
#!/bin/bash

for i in {1..20}
do
        python3 ~/python-tag-scraper/main.py
        sleep 3
done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment