Skip to content

Instantly share code, notes, and snippets.

View lockefox's full-sized avatar

John Purcell lockefox

53 followers · 4 following
View GitHub Profile
@lockefox
lockefox / angel_npc-kill_price-stats_raw.csv
Created February 4, 2016 16:28
Raw Angel_NPC-KILL and CREST price data, EVE Online
Date Cynabal Dramiel Machariel SUM_factionKills mean
2016-01-12 166254000 48290500 447648000 721379 2.207308e+08
2016-01-13 165430000 46586700 449224000 831691 2.204136e+08
2016-01-14 165570000 46734000 456961000 787412 2.230883e+08
2016-01-15 161172000 47407600 451482000 788098 2.200205e+08
2016-01-16 164873000 47727500 447546000 876720 2.200488e+08
2016-01-17 165724000 47626700 448990000 857619 2.207802e+08
2016-01-18 165303000 48373500 445440000 789442 2.197055e+08
2016-01-19 166448000 45800300 442759000 743316 2.183358e+08
2016-01-20 174764000 48519400 439389000 790141 2.208908e+08
@lockefox
lockefox / angel_npc-kill_price-stats_proc.csv
Created February 4, 2016 16:36
We can make this file beautiful and searchable if this error is corrected: It looks like row 6 should actually have 25 columns, instead of 14. in line 5.
Date,Cynabal,Dramiel,Machariel,SUM_factionKills,mean,log_activity,ln_activity,sqrt_activity,activity_shift1,activity_shift2,activity_shift3,activity_shift4,activity_shift5,ln-activity_shift1,ln-activity_shift2,ln-activity_shift3,ln-activity_shift4,ln-activity_shift5,5d_movingAvg_cyn,5d_deviation_cyn,5d_movingAvg_dram,5d_deviation_dram,5d_movingAvg_mach,5d_deviation_mach
1/12/2016,166254000,48290500,447648000,721379,220730800,5.858163495,13.48891994,849.3403323,,,,,,,,,,,,,,,,
1/13/2016,165430000,46586700,449224000,831691,220413600,5.919962002,13.63121626,911.9709425,721379,,,,,13.48891994,,,,,,,,,,
1/14/2016,165570000,46734000,456961000,787412,223088300,5.896202029,13.5765069,887.3623837,831691,721379,,,,13.63121626,13.48891994,,,,,,,,,
1/15/2016,161172000,47407600,451482000,788098,220020500,5.896580225,13.57737773,887.7488384,787412,831691,721379,,,13.5765069,13.63121626,13.48891994,,,,,,,,
1/16/2016,164873000,47727500,447546000,876720,220048800,5.942860914,13.68394295,936.333274,788098,787412,831691,721379,
@lockefox
lockefox / skill_trading_plot.r
Last active February 18, 2016 13:24
PROSPER Skill Trade/RMT Plotting script
library(RODBC)
library(ggplot2)
library(grid)
library(reshape)
library(scales)
library(data.table)
library(quantmod)
library(jsonlite)
library(cowplot)
library(plyr)
@lockefox
lockefox / crest_history_prophet.R
Last active April 16, 2017 13:50
packages.list <- c(
"data.table",
"quantmod",
"jsonlite",
"prophet",
"dplyr",
"ggplot2",
"ggthemes",
"cowplot",
"zoo"
@lockefox
lockefox / stock_prophet_experiment.R
Last active February 27, 2017 19:17
#### Libraries ####
library(quantmod) #getSymbols/chartSeries
library(zoo) #rollmean/rollapply
library(plyr)
library(TTR)
library(prophet)
library(dplyr)
library(ggplot2)
library(ggthemes)
library(cowplot)
@lockefox
lockefox / fetch_zkb_solo.py
Last active August 19, 2017 16:54
zkb vs pandas demo
import requests
import pymongo
req = requests.get('https://zkillboard.com/api/losses/solo/')
req.raise_for_status()
recent_solo_losses_raw = req.json() #200 latest events
conn = pymongo.MongoClient(MONGO_CONNECTION_STRING)
conn.db.zkb.insert(recent_solo_losses_raw)
@lockefox
lockefox / sample_km.json
Created August 19, 2017 16:41
{
"killID": 64158479,
"solarSystemID": 30000157,
"killTime": "2017-08-19 16:28:48",
"moonID": 0,
"victim": {
"shipTypeID": 583,
"characterID": 94315372,
"characterName": "Canatana Jovakko",
"corporationID": 1699307293,
@lockefox
lockefox / dict_query.py
Created August 19, 2017 17:10
import pymongo
import pandas as pd
conn = pymongo.MongoClient(MONGO_CONNECTION_STRING)
exclude_data = {'_id': False, 'attackers': False, 'items': False}
raw_data = list(conn.db.zkb.find({}, projection=exclude_data))
kill_df = pd.DataFrame(raw_data)
@lockefox
lockefox / cleanup.py
Created August 19, 2017 17:39
## Split out JSON-blob columns into their own tables ##
# Rotates out key/value pairs into their own columns
position_df = pd.DataFrame(list(kill_df['position']))
victim_df = pd.DataFrame(list(kill_df['victim']))
zkb_df = pd.DataFrame(list(kill_df['zkb']))
## Merge pivoted data back into source frame ##
# row-index has not changed, append-horizontally
combined_df = pd.concat([kill_df, position_df, victim_df, zkb_df], axis=1)
@lockefox
lockefox / pivot_list_concat.py
Last active August 22, 2017 19:34
data_columns = ['killID', 'solarSystemID', 'killTime', 'moonID']
pivot_column = 'items'
index_column = 'killID'
## Make source DataFrame ##
raw_df = pd.DataFrame(raw_data)
## Row-by-row rotate and append desired data ##
result_df = None
for row in raw_df.itertuples():