Skip to content

Instantly share code, notes, and snippets.

View mxchinegod's full-sized avatar
💭
🐛fixing

Dylan mxchinegod

💭
🐛fixing
View GitHub Profile
@mxchinegod
mxchinegod / sec_ripper.py
Created June 18, 2023 13:41
Disgusting SEC Filing ripper
import requests
from tqdm import tqdm
import feedparser
from bs4 import BeautifulSoup
from time import sleep
import os
from pprint import pprint
#!python3 -m pip install feedparser
def fancy_message(tag, body):
@mxchinegod
mxchinegod / attention_patterns.py
Last active June 18, 2023 13:42
A token matrix visualization representing the importance a token has in multi-headed attention layers in transformers like GPT-2
import torch
from transformers import GPT2Tokenizer, GPT2Model
import matplotlib.pyplot as plt
import numpy as np
from matplotlib.animation import FuncAnimation
def get_model_info(model):
num_layers = model.config.n_layer
num_heads = model.config.n_head
hidden_dim = model.config.n_embd
@mxchinegod
mxchinegod / fancy_message.py
Last active June 8, 2023 14:09
Convenient Error wrapper for any Python code where supervised error context is strings
def fancy_message(tag, body):
tags = [
("FATAL", "☠️", "\033[91m"), # Red color for FATAL
("WARN", "🚨", "\033[93m"), # Yellow color for WARN
("INFO", "ℹ️", "\033[94m"), # Blue color for INFO
("WAIT", "☕️", "\033[96m") # Cyan color for WAIT
]
matching_tags = [x for x in tags if x[0] == tag.upper()]
if matching_tags:
@mxchinegod
mxchinegod / FOMC_Reserve_Transcript_Sentiment.py
Last active June 7, 2023 13:44
Uses AutoModel to do sentence (sequence) classification on FOMC reports using a sentiment model specific to financial data
import requests
import io
from pdfminer.pdfinterp import PDFResourceManager, PDFPageInterpreter
from pdfminer.converter import TextConverter
from pdfminer.layout import LAParams
from pdfminer.pdfpage import PDFPage
import re
import torch
import termcolor
from transformers import AutoTokenizer, AutoModel, AutoModelForSequenceClassification
@mxchinegod
mxchinegod / .pre-commit-config.yaml
Created January 27, 2023 13:04
a basic cleanup pre-commit configuration on the cloud
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v2.3.0
hooks:
- id: check-yaml
- id: end-of-file-fixer
- id: trailing-whitespace
- id: mixed-line-ending
- repo: https://github.com/psf/black
rev: 22.10.0
@mxchinegod
mxchinegod / commit-msg
Last active January 27, 2023 18:12
commit-msg git hook to force Angular conventions!
#!/bin/bash
REGEX="(feat|fix|docs|ui|chore|locale)\((refactor|perf|workflow|build|CI|typos|tests|types|wip|release|dep)\):\s.+"
if ! grep -iqE "$REGEX" "$1"; then
echo "Invalid commit message format. Use the following format: [emoji] [revert: ?]<type>[(scope)?]: <message>"
echo "e.g. '🎨 feat(workflow): Added commit-msg hook'"
exit 1
else
pre-commit
fi
@mxchinegod
mxchinegod / xlsx2doc.py
Last active January 26, 2023 01:23
Python script to take rows of an Excel file and put them into a Word Document, by page.
import openpyxl
from docx import Document
def stupid_tax(x):
try:
x = x.split('.')[0]
return x
except:
return x