Parsing Networks in the TNPM format for consumption in AequilibraE

tnpm_parsing.py

import aequilibrae
import os
import sys
import numpy as np
import pandas as pd
import openmatrix as omx
from aequilibrae.matrix import AequilibraeMatrix

data_folder = 'path/to/the/anaheim/folder'

matfile = os.path.join(data_folder, 'Anaheim_trips.tntp')

# Creating the matrix
f = open(matfile, 'r')
all_rows = f.read()
blocks = all_rows.split('Origin')[1:]
matrix = {}
for k in range(len(blocks)):
    orig = blocks[k].split('\n')
    dests = orig[1:]
    orig=int(orig[0])

    d = [eval('{'+a.replace(';',',').replace(' ','') +'}') for a in dests]
    destinations = {}
    for i in d:
        destinations = {**destinations, **i}
    matrix[orig] = destinations
zones = max(matrix.keys())
mat = np.zeros((zones, zones))
for i in range(zones):
    for j in range(zones):
        mat[i, j] = matrix[i+1].get(j+1,0)

# Saving the matrix to an OMX container
omxfile = matfile.replace('tntp', 'omx')
index = np.arange(zones) + 1
myfile = omx.open_file(omxfile,'w')
myfile['matrix'] = mat
myfile.create_mapping('taz', index)
myfile.close()

# Or if you prefer an AequilibraE matrix
aemfile = matfile.replace('tntp', 'aem')
aem = AequilibraeMatrix()
kwargs = {'file_name': aemfile,
          'zones': zones,
          'matrix_names': ['matrix']}

aem.create_empty(**kwargs)
aem.matrix['matrix'][:,:] = mat[:,:]
aem.index[:] = index[:]

# Now let's parse the network
net = os.path.join(data_folder, 'Anaheim_net.tntp')
net = pd.read_csv(net, skiprows=7, sep='\t')

# If you want to create an AequilibraE matrix for computation, then it follows
from aequilibrae.paths import Graph
g = Graph()
g.cost = net['Free Flow Time (min)'].values
g.capacity = net['Capacity (veh/h)'].values
g.free_flow_time = net['Free Flow Time (min)'].values

network = net[['Tail', 'Head', 'Free Flow Time (min)', 'Capacity (veh/h)' ]]
network.columns = ['a_node', 'b_node', 'time', 'capacity']
network = network.assign(direction=1)
g.network = network.to_records(index=False)
g.network_ok = True
g.status = 'OK'
g.prepare_graph(index)

A few good practice tips, avoid reusing variable names orig as shown in orig=int(orig[0])

Also always avoid using eval especially here where it is reading a file https://nedbatchelder.com/blog/201206/eval_really_is_dangerous.html

Having used an amended version of the script, how can I then save this data to a project?

@alex, the TNPM data is a trusted source, so I am not really concerned with this type of vulnerability. On your question... It does not make sense to convert the graph into a project, as geographical information is missing. The only two TNPM instances that I found to have the necessary info for that were Sioux Falls and Chicago Regional

Ah thanks I was misunderstanding the usage of the project, I was thinking it as an overall container for a model rather than specifically for storing GIS of your network.

I'm trying to get the example code in the docs to work without starting with any pre-generated files as is currently the case for the Sioux Falls and Chicago Examples which have a bunch of files in all the folders.

Running the graph created from this data through the example in the documents, I was getting
res.path returning an array of zeros. I had to add a line

network["link_id"] = network.index.values.astype(int)

for res.path to return anything useful, browsing the code for the Graph class it looked like link_id and id are required fields so I'm not sure how the code was allowing a graph to be generated without them? Secondly I'm not sure what the purpose of id given the graph seems to be structured as an array of links?

Hi Alex,
The project IS to hold the overall project, but the network is a required component of it. The graph, which is what this gist helps producing is just an artifact used for computational purposes.

The link_id corresponds to a link that can be bi-directional, while the id in the graph is necessarily directional. The fid in the SQLite file exists due to requirements on the SpatiaLite side.

Long-story-short, it may help you to think of AequilibraE like you would of any commercial modeling package (e.g. TransCAD, Emme, Visum), The purposes and use cases are about the same. I can give you a better rundown if you tell me exactly what your objectives and use cases for the package are.

I've used and have had training in EMME and VISUM but a little rusty when it comes to them.
I'm looking to see if the aequilibrae back-end is a suitable library for pipe-lining a process for quickly generating and running a simple 4-stage sketch model and making it reproducible for more than one location. From reading the docs it looks like it should be able to help with that and I'm just trying to follow the examples to better understand the exact capabilities.

This is what AequilibraE has been designed to do, but it still has a long way to go if compared to the commercial platforms you mentioned.