Skip to content

Instantly share code, notes, and snippets.

John Purcell lockefox

Block or report user

Report or block lockefox

Hide content and notifications from this user.

Learn more about blocking users

Contact Support about this user’s behavior.

Learn more about reporting abuse

Report abuse
View GitHub Profile
@lockefox
lockefox / python_logging_blog.md
Last active Aug 31, 2019
Everything You Never Wanted To Know About Python Logging (1)
View python_logging_blog.md

Everything You Never Wanted To Know About Python Logging

print() can't remain the only tool for debugging/reporting in programs. Python offers an extremely powerful suite of reporting tools with the logging library. But be warned, logging isn't merely "print() on steroids", and mishandling this library can become a major pain point.

In this guide we will discuss the underlying model behind logging and how to get the most out of the reporting in our Python projects.

The Life of a Logging Message

TODO: grafix

@lockefox
lockefox / config_utils.py
Created Nov 19, 2018
Old code removed from ProsperCommon and pytest-prosper because it's weird and bad
View config_utils.py
'''utilities.py: worker functions for CREST calls'''
from os import path
import logging
from datetime import datetime
from prosper.common.prosper_config import get_config, get_local_config_filepath
DEFAULT_LOGGER = logging.getLogger('NULL')
DEFAULT_LOGGER.addHandler(logging.NullHandler())
@lockefox
lockefox / eve_online_map_experiments.ipynb
Created Oct 31, 2017
Searching for patterns in EVE Online map statistics
View eve_online_map_experiments.ipynb
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@lockefox
lockefox / coin_demo_doge.ipynb
Created Oct 3, 2017
CryptoCompare history [DOGE] + fbprophet forecasting
View coin_demo_doge.ipynb
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@lockefox
lockefox / stock_demo.ipynb
Created Sep 29, 2017
Pandas-Datareader + fbprophet forecasting
View stock_demo.ipynb
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@lockefox
lockefox / coin_demo.ipynb
Last active Sep 29, 2017
CryptoCompare history + fbprophet forecasting
View coin_demo.ipynb
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
View pivot_list_melt.py
data_columns = ['killID', 'solarSystemID', 'killTime', 'moonID']
pivot_column = 'items'
index_column = 'killID'
## Make source DataFrame ##
raw_df = pd.DataFrame(raw_data)
## Rotate list-column into its own frame ##
pivot_df = pd.DataFrame(list(raw_df[pivot_column]))
View pivot_results.csv
Data Scenario 200 records 2000 records 50000 records
Pivot Dicts - Pandas 0.015s 0.041s 0.804s
Pivot Dicts - Raw Python 0.028s 0.217s 5.781s
Pivot Lists - Pandas (concat) 0.834s 9.672 2069.694s
Pivot Lists - Pandas (melt) 0.029s 0.190s 17.454s
Pivot Lists - Raw Python 0.043s 0.213s 5.502s
View pivot_list_concat.py
data_columns = ['killID', 'solarSystemID', 'killTime', 'moonID']
pivot_column = 'items'
index_column = 'killID'
## Make source DataFrame ##
raw_df = pd.DataFrame(raw_data)
## Row-by-row rotate and append desired data ##
result_df = None
for row in raw_df.itertuples():
View cleanup.py
## Split out JSON-blob columns into their own tables ##
# Rotates out key/value pairs into their own columns
position_df = pd.DataFrame(list(kill_df['position']))
victim_df = pd.DataFrame(list(kill_df['victim']))
zkb_df = pd.DataFrame(list(kill_df['zkb']))
## Merge pivoted data back into source frame ##
# row-index has not changed, append-horizontally
combined_df = pd.concat([kill_df, position_df, victim_df, zkb_df], axis=1)
You can’t perform that action at this time.