Skip to content

Instantly share code, notes, and snippets.

View chris-lovejoy's full-sized avatar

ChrisLovejoy chris-lovejoy

View GitHub Profile
@chris-lovejoy
chris-lovejoy / job_scraper.py
Last active October 5, 2020 16:20
single function to call
def find_jobs_from(website, job_title, location, desired_characs, filename="results.xls"):
"""
This function extracts all the desired characteristics of all new job postings
of the title and location specified and returns them in single file.
The arguments it takes are:
- Website: to specify which website to search (options: 'Indeed' or 'CWjobs')
- Job_title
- Location
- Desired_characs: this is a list of the job characteristics of interest,
from titles, companies, links and date_listed.
@chris-lovejoy
chris-lovejoy / yt_api_call.py
Created November 21, 2020 21:32
Call the YouTube API
# Call the YouTube API
api_key = ‘AIzpSyAq3L9DiPK0KxrGBbdY7wNN7kfPbm_hsPg’ # Enter your own API key – this one won’t work
youtube_api = build(‘youtube’, ‘v3’, developerKey = api_key)
results = youtube_api.search().list(q=search_terms, part=’snippet’, type=’video’,
order=’viewCount’, maxResults=50).execute()
@chris-lovejoy
chris-lovejoy / published_at.py
Created November 21, 2020 21:34
Select date published at
publishedAt = results[‘items’][0][‘snippet’][‘publishedAt’]
@chris-lovejoy
chris-lovejoy / view_to_sub_ratio.py
Created November 21, 2020 21:36
Function to calculate view-to-sub ratio
# Function to calculate view-to-sub ratio
def view_to_sub_ratio(viewcount, num_subscribers):
if num_subscribers == 0:
return 0
else:
ratio = viewcount / num_subscribers
return ratio
@chris-lovejoy
chris-lovejoy / restrict_search_period.py
Last active November 21, 2020 21:42
Restricting search period
# Function to create string for search start date
def get_start_date_string(search_period_days):
"""Returns string for date at start of search period."""
search_start_date = datetime.today() – timedelta(search_period_days)
date_string = datetime(year=search_start_date.year,month=search_start_date.month,
day=search_start_date.day).strftime(‘%Y-%m-%dT%H:%M:%SZ’)
return date_string
# Creating date string and executing search
date_string = get_start_date_string(7)
@chris-lovejoy
chris-lovejoy / score_div_days.py
Created November 21, 2020 21:43
Adding days since published into custom score
# Adding days since published into custom score
def custom_score(viewcount, ratio, days_since_published):
if ratio > 5:
ratio = 5
score = (viewcount * ratio) / days_since_published
return score
@chris-lovejoy
chris-lovejoy / ratio_restricted.py
Last active November 22, 2020 21:45
Calculating ratio while removing edge cases (of low views or low subscribers)
# Calculating ratio while removing edge cases (of low views or low subscribers)
def custom_score(viewcount, ratio, days_since_published):
ratio = min(ratio, 5)
score = (viewcount * ratio)
return score
@chris-lovejoy
chris-lovejoy / video_finder.py
Last active March 8, 2021 21:59
Initial search
from apiclient.discovery import build
api_key='AIzaKyAq3L9BiVO0PXrGBhhY0cNN9fkPmm_BsPg' # (not a real API key)
youtube_api = build('youtube','v3', developerKey=api_key)
# Search videos
video_search_results = youtube_api.search().list(q='productivity', part='snippet', type='video',
order='viewCount', maxResults=50).execute()
# Search channels
@chris-lovejoy
chris-lovejoy / job_scraper.py
Created May 1, 2020 13:15
use driver to extract HTML soup
def make_job_search(job_title, location, driver):
driver.get('https://www.cwjobs.co.uk/')
# Select the job box
job_title_box = driver.find_element_by_name('Keywords')
# Send job information
job_title_box.send_keys(job_title)
# Selection location box