Skip to content

Instantly share code, notes, and snippets.

@fscottfoti
Last active August 29, 2015 14:05
Show Gist options
  • Save fscottfoti/68efce874b9efb5bebcd to your computer and use it in GitHub Desktop.
Save fscottfoti/68efce874b9efb5bebcd to your computer and use it in GitHub Desktop.
This is the code required to convert the network from the old pickled format to just storing the networks as dataframes in an hdf5 file (pandas style)
import pandas as pd
import cPickle
import sys
args = sys.argv[1:]
assert len(args) == 2, "go.py <infile.jar> <outfile.h5>"
d = cPickle.load(open(args[0]))
nodes = pd.DataFrame(d['nodes'], index=d['nodeids'])
nodes.columns = ['x', 'y']
edges = pd.DataFrame(d['edges'], index=d['edgeids'])
edges.columns = ['from', 'to']
edges['weight'] = d['edgeweights']
for col in ['from', 'to']:
# not pretty - pandas isn't really designed for using offsets
edges[col] = nodes.reset_index().iloc[edges[col]]["index"].values
print edges.describe()
# test is out
import pandana as pdna
pdna.Network(nodes["x"], nodes["y"], edges["from"], edges["to"],
edges[["weight"]])
# write it out
store = pd.HDFStore(args[1])
store["nodes"] = nodes
store["edges"] = edges
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment