Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Parsing Networks in the TNPM format for consumption in AequilibraE
import aequilibrae
import os
import sys
import numpy as np
import pandas as pd
import openmatrix as omx
from aequilibrae.matrix import AequilibraeMatrix
data_folder = 'path/to/the/anaheim/folder'
matfile = os.path.join(data_folder, 'Anaheim_trips.tntp')
# Creating the matrix
f = open(matfile, 'r')
all_rows = f.read()
blocks = all_rows.split('Origin')[1:]
matrix = {}
for k in range(len(blocks)):
orig = blocks[k].split('\n')
dests = orig[1:]
orig=int(orig[0])
d = [eval('{'+a.replace(';',',').replace(' ','') +'}') for a in dests]
destinations = {}
for i in d:
destinations = {**destinations, **i}
matrix[orig] = destinations
zones = max(matrix.keys())
mat = np.zeros((zones, zones))
for i in range(zones):
for j in range(zones):
mat[i, j] = matrix[i+1].get(j+1,0)
# Saving the matrix to an OMX container
omxfile = matfile.replace('tntp', 'omx')
index = np.arange(zones) + 1
myfile = omx.open_file(omxfile,'w')
myfile['matrix'] = mat
myfile.create_mapping('taz', index)
myfile.close()
# Or if you prefer an AequilibraE matrix
aemfile = matfile.replace('tntp', 'aem')
aem = AequilibraeMatrix()
kwargs = {'file_name': aemfile,
'zones': zones,
'matrix_names': ['matrix']}
aem.create_empty(**kwargs)
aem.matrix['matrix'][:,:] = mat[:,:]
aem.index[:] = index[:]
# Now let's parse the network
net = os.path.join(data_folder, 'Anaheim_net.tntp')
net = pd.read_csv(net, skiprows=7, sep='\t')
# If you want to create an AequilibraE matrix for computation, then it follows
from aequilibrae.paths import Graph
g = Graph()
g.cost = net['Free Flow Time (min)'].values
g.capacity = net['Capacity (veh/h)'].values
g.free_flow_time = net['Free Flow Time (min)'].values
network = net[['Tail', 'Head', 'Free Flow Time (min)', 'Capacity (veh/h)' ]]
network.columns = ['a_node', 'b_node', 'time', 'capacity']
network = network.assign(direction=1)
g.network = network.to_records(index=False)
g.network_ok = True
g.status = 'OK'
g.prepare_graph(index)
@alexdawn

This comment has been minimized.

Copy link

alexdawn commented Mar 19, 2020

A few good practice tips, avoid reusing variable names orig as shown in orig=int(orig[0])

Also always avoid using eval especially here where it is reading a file https://nedbatchelder.com/blog/201206/eval_really_is_dangerous.html

Having used an amended version of the script, how can I then save this data to a project?

@pedrocamargo

This comment has been minimized.

Copy link
Owner Author

pedrocamargo commented Mar 19, 2020

@alex, the TNPM data is a trusted source, so I am not really concerned with this type of vulnerability. On your question... It does not make sense to convert the graph into a project, as geographical information is missing. The only two TNPM instances that I found to have the necessary info for that were Sioux Falls and Chicago Regional

@alexdawn

This comment has been minimized.

Copy link

alexdawn commented Mar 30, 2020

Ah thanks I was misunderstanding the usage of the project, I was thinking it as an overall container for a model rather than specifically for storing GIS of your network.

I'm trying to get the example code in the docs to work without starting with any pre-generated files as is currently the case for the Sioux Falls and Chicago Examples which have a bunch of files in all the folders.

Running the graph created from this data through the example in the documents, I was getting
res.path returning an array of zeros. I had to add a line

network["link_id"] = network.index.values.astype(int)

for res.path to return anything useful, browsing the code for the Graph class it looked like link_id and id are required fields so I'm not sure how the code was allowing a graph to be generated without them? Secondly I'm not sure what the purpose of id given the graph seems to be structured as an array of links?

@pedrocamargo

This comment has been minimized.

Copy link
Owner Author

pedrocamargo commented Mar 30, 2020

Hi Alex,
The project IS to hold the overall project, but the network is a required component of it. The graph, which is what this gist helps producing is just an artifact used for computational purposes.

The link_id corresponds to a link that can be bi-directional, while the id in the graph is necessarily directional. The fid in the SQLite file exists due to requirements on the SpatiaLite side.

Long-story-short, it may help you to think of AequilibraE like you would of any commercial modeling package (e.g. TransCAD, Emme, Visum), The purposes and use cases are about the same. I can give you a better rundown if you tell me exactly what your objectives and use cases for the package are.

@alexdawn

This comment has been minimized.

Copy link

alexdawn commented Mar 31, 2020

I've used and have had training in EMME and VISUM but a little rusty when it comes to them.
I'm looking to see if the aequilibrae back-end is a suitable library for pipe-lining a process for quickly generating and running a simple 4-stage sketch model and making it reproducible for more than one location. From reading the docs it looks like it should be able to help with that and I'm just trying to follow the examples to better understand the exact capabilities.

@pedrocamargo

This comment has been minimized.

Copy link
Owner Author

pedrocamargo commented Mar 31, 2020

This is what AequilibraE has been designed to do, but it still has a long way to go if compared to the commercial platforms you mentioned.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.