Skip to content

Instantly share code, notes, and snippets.

David Yerrington dyerrington

View GitHub Profile
View sf_slicing_apply_map.ipynb
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
View parse_jupyter.md

Parse Jupyter

This is a basic class that makes it convenient to parse notebooks. I built a larger version of this that was used for clustering documents to create symantic indeices that linked related content together for a personal project. You can use this to parse notebooks for doing things like NLP or preprocessing.

Usage

parser = ParseJupyter("./Untitled.ipynb")
parser.get_cells(source_only = True, source_as_string = True)
View machine_learning_flashcards.py
import tweepy
import wget
import os
oauth = {
"consumer_key": "",
"consumer_secret": ""
}
access = {
View sf_review.ipynb
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@dyerrington
dyerrington / my_little_pony_lstm.py
Created Jul 16, 2018
As a point of comparison with the default Nietzsche example from the Keras repo, this little experiment swaps out the dataset with forum comments from My Little Pony subreddit.
View my_little_pony_lstm.py
'''Example script to generate text from Nietzsche's writings.
At least 20 epochs are required before the generated text
starts sounding coherent.
It is recommended to run this script on GPU, as recurrent
networks are quite computationally intensive.
If you try this script on new data, make sure your corpus
has at least ~100k characters. ~1M is better.
'''
from __future__ import print_function
@dyerrington
dyerrington / custom_vectorizer.py
Created Jul 12, 2018
Remove "n-grams" first, before stopwords with this handy class that extends the functionality of scikit-learn's CountVectorizer. Substitute the class extension for other types of vectorizers such as TfIDF in the class definition at the top.
View custom_vectorizer.py
# defines a custom vectorizer class
class CustomVectorizer(CountVectorizer):
stop_grams = []
def __init__(self, stop_grams = [], **opts):
self.stop_grams = stop_grams
super().__init__(**opts)
def remove_ngrams(self, doc):
@dyerrington
dyerrington / readme.md
Last active Jan 9, 2019
This is a very basic data generator to test recommender systems. A future version may simulate the actual sparseness of ratings data with a simple bootstrap function but for now, numpy generator does the job.
View readme.md

RecData

To use this snippet, install faker:

pip install faker
View dsi_student_install_guide.md
@dyerrington
dyerrington / environment.yml
Last active Apr 19, 2018
This assumes you've created an environment called "dsi". To use this file, simply click on "raw" then download the contents to a new file on your local system called "environment".
View environment.yml
name: dsi
channels:
- conda-forge
- defaults
dependencies:
- appnope=0.1.0=py36_0
- asn1crypto=0.22.0=py36_0
- attrs=17.2.0=py_1
- automat=0.6.0=py36_0
- backports=1.0=py36_1
@dyerrington
dyerrington / environment.yml
Created Apr 19, 2018
This assumes you've created an environment called "dsi", like this:
View environment.yml
name: dsi
channels:
- conda-forge
- defaults
dependencies:
- asn1crypto=0.22.0=py36_0
- beautifulsoup4=4.5.3=py36_0
- blas=1.1=openblas
- bleach=2.0.0=py36_0
- bokeh=0.12.9=py36_0
You can’t perform that action at this time.