This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
PROJECT_NAME := $(shell basename $CURDIR) | |
VIRTUAL_ENVIRONMENT := $(CURDIR)/.venv | |
LOCAL_PYTHON := $(VIRTUAL_ENVIRONMENT)/bin/python3 | |
define HELP | |
Manage $(PROJECT_NAME). Usage: | |
make run - Run $(PROJECT_NAME) locally. | |
make dev - Run $(PROJECT_NAME) in development mode. | |
make install - Create local virtualenv & install dependencies. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
"""Extract nested values from a JSON tree.""" | |
def json_extract(obj, key): | |
"""Recursively fetch values from nested JSON.""" | |
arr = [] | |
def extract(obj, arr, key): | |
"""Recursively search for values of key in JSON tree.""" | |
if isinstance(obj, dict): |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
"""Script configuration.""" | |
from os import path | |
BASE_DIR = path.abspath(path.dirname(__file__)) | |
# CSV Exported from `deadlinkchecker` | |
CSV_EXPORTED_BROKEN_LINKS = f"{BASE_DIR}/data/brokenlinks.csv" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
"""Decorator to determine execution time of a given function.""" | |
from time import time | |
def timeit(method: func): | |
"""Print execution time of decorated function.""" | |
def timed(*args, **kw): | |
ts = time() | |
result = method(*args, **kw) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
"""Find Symmetric Differences between two Pandas DataFrames.""" | |
def dataframe_difference(df1, df2, which=None): | |
"""Find rows which are different.""" | |
comparison_df = df1.merge( | |
df2, | |
indicator=True, | |
how='outer' | |
) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
PROJECT_NAME := $(shell basename $CURDIR) | |
VIRTUAL_ENVIRONMENT := $(CURDIR)/.venv | |
LOCAL_PYTHON := $(VIRTUAL_ENVIRONMENT)/bin/python3 | |
PROJECT_ENTRY_POINT := $(shell $CURDIR)/main.py | |
define HELP | |
Manage $(PROJECT_NAME). Usage: | |
make run - Run $(PROJECT_NAME) locally. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import sys | |
import pymysql | |
import logging | |
class Database: | |
"""Database connection class.""" | |
def __init__(self, config): | |
self.host = config.db_host |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import os | |
import json | |
import glob | |
import PIL | |
from PIL import Image | |
def get_all_images(): | |
"""Create an array of PNGs and JPGs.""" | |
img_arr = [] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/bin/bash | |
PROJECT=$(pwd) | |
CURRENT_GHOST_VERSION=$(ghost version) | |
NEW_GHOST_VERSION=$(ghost check-update) | |
# Ensure latest NPM & Ghost CLI versions | |
echo "Getting latest version of NPM & Ghost-ClI" | |
sudo npm install -g npm@latest | |
sudo npm install -g ghost-cli@latest |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import os | |
import json | |
import boto3 | |
from botocore.client import Config | |
import botocore | |
from urllib.parse import unquote | |
import PIL | |
# Initialize a session using DigitalOcean Spaces. |
NewerOlder