Skip to content

Instantly share code, notes, and snippets.

@knaaptime

knaaptime/UrbanSim test 2

Created January 17, 2014 21:51
Show Gist options
  • Save knaaptime/8482398 to your computer and use it in GitHub Desktop.
Save knaaptime/8482398 to your computer and use it in GitHub Desktop.
Download ZIP
UrbanSim run using foti branch
Raw
UrbanSim test 2
Zero:example knaaptime$ ./run.sh
Fri Jan 17 16:48:14 2014
Running buildings.json
Fetching buildings
Fetching parcels
Fetching modify_table
Fetching jobs
Fetching modify_table
Fetching modify_table
Specifying model in 1.004534
Finished executing in 1.004562 seconds
Specifying model in 0.959697
Finished executing in 0.962592 seconds
SIMULATED buildings.json model in 0.963 seconds
Running hhlds.json
Fetching households
Fetching modify_table
Specifying model in 0.788763
Finished executing in 0.790800 seconds
Specifying model in 0.761408
Finished executing in 0.763465 seconds
SIMULATED hhlds.json model in 0.764 seconds
Running jobs.json
Specifying model in 0.334225
Finished executing in 0.335192 seconds
Specifying model in 0.378522
Finished executing in 0.379146 seconds
SIMULATED jobs.json model in 0.379 seconds
Running zones.json
Fetching zones
Fetching modify_table
Fetching travel_data
Fetching modify_table
Specifying model in 0.886182
Finished executing in 0.886209 seconds
Specifying model in 0.832673
Finished executing in 0.832709 seconds
SIMULATED zones.json model in 0.833 seconds
Running rsh.json
Done merging land use and choosers in 0.083303
Finished specifying in 0.185821 seconds
Specifying model in 0.203870
Estimating hedonic for 1 with 297376 observations
historic new year_built ln_parcel_acres \
count 297376.000000 297376.000000 297376.000000 2.973760e+05
mean 0.070934 0.365013 1980.371378 4.129644e-01
std 0.256715 0.481435 19.725919 7.159198e-01
min 0.000000 0.000000 1790.000000 2.149406e-14
25% 0.000000 0.000000 1967.000000 1.334722e-01
50% 0.000000 0.000000 1984.000000 1.765473e-01
75% 0.000000 1.000000 1996.000000 3.790199e-01
max 1.000000 1.000000 2012.000000 1.126495e+01
ln_sqft_per_unit ln_average_income ln_population_in_range \
count 297376.000000 297376.000000 297376.000000
mean 7.366880 10.959103 11.884783
std 0.409320 1.167037 1.437795
min 4.615120 0.000000 0.000000
25% 7.029973 10.866889 11.707925
50% 7.346655 11.080326 12.381000
75% 7.630947 11.304999 12.800691
max 12.288689 11.819008 13.235408
ln_time_to_downtown const
count 297376.000000 297376
mean 3.340930 1
std 0.463769 0
min 2.051068 1
25% 3.031756 1
50% 3.228030 1
75% 3.745288 1
max 4.637399 1
[8 rows x 9 columns]
OLS Regression Results
==================================================================================
Dep. Variable: unit_price_residential R-squared: 0.491
Model: OLS Adj. R-squared: 0.491
Method: Least Squares F-statistic: 3.579e+04
Date: Fri, 17 Jan 2014 Prob (F-statistic): 0.00
Time: 16:48:22 Log-Likelihood: -1.6818e+05
No. Observations: 297376 AIC: 3.364e+05
Df Residuals: 297367 BIC: 3.365e+05
Df Model: 8
==========================================================================================
coef std err t P>|t| [95.0% Conf. Int.]
------------------------------------------------------------------------------------------
historic 0.0024 0.004 0.578 0.563 -0.006 0.011
new -0.0614 0.003 -22.693 0.000 -0.067 -0.056
year_built 0.0043 8.66e-05 49.795 0.000 0.004 0.004
ln_parcel_acres 0.0381 0.001 30.304 0.000 0.036 0.041
ln_sqft_per_unit 0.9179 0.002 447.874 0.000 0.914 0.922
ln_average_income 0.0821 0.001 120.506 0.000 0.081 0.083
ln_population_in_range -0.0114 0.001 -11.432 0.000 -0.013 -0.009
ln_time_to_downtown -0.1591 0.003 -47.906 0.000 -0.166 -0.153
const -3.3623 0.165 -20.318 0.000 -3.687 -3.038
==============================================================================
Omnibus: 140602.129 Durbin-Watson: 0.751
Prob(Omnibus): 0.000 Jarque-Bera (JB): 2822495.447
Skew: 1.804 Prob(JB): 0.00
Kurtosis: 17.655 Cond. No. 4.20e+05
==============================================================================
Warnings:
[1] The condition number is large, 4.2e+05. This might indicate that there are
strong multicollinearity or other numerical problems.
Specifying model in 0.002667
Estimating hedonic for 2 with 4711 observations
historic new ln_average_income const
count 4711.000000 4711.000000 4711.000000 4711
mean 0.167268 0.060921 10.171841 1
std 0.373255 0.239211 2.354934 0
min 0.000000 0.000000 0.000000 1
25% 0.000000 0.000000 10.445782 1
50% 0.000000 0.000000 10.637335 1
75% 0.000000 0.000000 10.947579 1
max 1.000000 1.000000 11.685293 1
[8 rows x 4 columns]
OLS Regression Results
==================================================================================
Dep. Variable: unit_price_residential R-squared: 0.038
Model: OLS Adj. R-squared: 0.037
Method: Least Squares F-statistic: 61.61
Date: Fri, 17 Jan 2014 Prob (F-statistic): 4.59e-39
Time: 16:48:22 Log-Likelihood: -5937.2
No. Observations: 4711 AIC: 1.188e+04
Df Residuals: 4707 BIC: 1.191e+04
Df Model: 3
=====================================================================================
coef std err t P>|t| [95.0% Conf. Int.]
-------------------------------------------------------------------------------------
historic 0.1676 0.034 4.987 0.000 0.102 0.233
new 0.0101 0.052 0.193 0.847 -0.093 0.113
ln_average_income 0.0652 0.005 12.314 0.000 0.055 0.076
const 10.3836 0.055 188.025 0.000 10.275 10.492
==============================================================================
Omnibus: 905.683 Durbin-Watson: 1.580
Prob(Omnibus): 0.000 Jarque-Bera (JB): 5470.965
Skew: -0.779 Prob(JB): 0.00
Kurtosis: 8.044 Cond. No. 47.0
==============================================================================
Finished executing in 0.874971 seconds
Done merging land use and choosers in 0.097057
Finished specifying in 0.123426 seconds
Specifying model in 0.144677
Generating rents on 304040 buildings
Specifying model in 0.003176
Generating rents on 5453 buildings
Finished executing in 0.270766 seconds
SIMULATED rsh.json model in 0.407 seconds
Running zones2.json
Specifying model in 0.018208
Finished executing in 0.018249 seconds
Specifying model in 0.008324
Finished executing in 0.008362 seconds
SIMULATED zones2.json model in 0.008 seconds
Running new_hhlds.json
SIMULATED new_hhlds.json model in 0.029 seconds
Running hlcm.json
Done merging land use and choosers in 0.083397
Estimating parameters for segment = (1, 0.0), size = 1154
Specifying model in 0.070168
average_price ln_average_income ln_population_in_range \
count 115400.000000 115400.000000 115400.000000
mean 12.179990 10.933910 11.867161
std 0.433353 1.223486 1.479274
min 0.000000 0.000000 0.000000
25% 11.990229 10.843286 11.638156
50% 12.190934 11.072011 12.381000
75% 12.386533 11.294388 12.812449
max 14.974602 11.819008 13.235408
ln_time_to_downtown income X average_income
count 115400.000000 115400.000000
mean 3.337257 18.994125
std 0.476494 5.324946
min 0.000000 0.000000
25% 3.015932 20.042354
50% 3.227324 20.591179
75% 3.750210 20.974545
max 4.562793 21.904810
[8 rows x 5 columns]
Null Log-liklihood: -5314.366395
Log-liklihood at convergence: -4958.450623
Log-liklihood ratio: 0.066972
+-------------------------+-------------+--------+---------+--------------+
| Variables | Coefficient | Stderr | T-score | Significance |
+=========================+=============+========+=========+==============+
| average price | -1.060 | 0.100 | -10.840 | *** |
+-------------------------+-------------+--------+---------+--------------+
| ln average income | -0.050 | 0.120 | -0.460 | |
+-------------------------+-------------+--------+---------+--------------+
| ln population in range | -0.020 | 0.030 | -0.540 | |
+-------------------------+-------------+--------+---------+--------------+
| ln time to downtown | 0.070 | 0.100 | 0.690 | |
+-------------------------+-------------+--------+---------+--------------+
| income X average income | 0.140 | 0.010 | 22.620 | *** |
+-------------------------+-------------+--------+---------+--------------+
Estimating parameters for segment = (1, 1.0), size = 1603
Specifying model in 0.086343
average_price ln_average_income ln_population_in_range \
count 160300.000000 160300.000000 160300.000000
mean 12.180372 10.936894 11.861200
std 0.448161 1.218706 1.488862
min 0.000000 0.000000 0.000000
25% 11.991727 10.847899 11.638156
50% 12.191078 11.072707 12.381000
75% 12.392412 11.296932 12.812449
max 14.974602 11.819008 13.235408
ln_time_to_downtown income X average_income
count 160300.000000 160300.000000
mean 3.338447 21.292772
std 0.476729 2.321288
min 0.000000 0.000000
25% 3.020937 21.276612
50% 3.227871 21.550636
75% 3.750210 21.804018
max 4.562793 22.597957
[8 rows x 5 columns]
Null Log-liklihood: -7382.087808
Log-liklihood at convergence: -6876.669273
Log-liklihood ratio: 0.068466
+-------------------------+-------------+--------+---------+--------------+
| Variables | Coefficient | Stderr | T-score | Significance |
+=========================+=============+========+=========+==============+
| average price | -0.750 | 0.100 | -7.120 | *** |
+-------------------------+-------------+--------+---------+--------------+
| ln average income | -0.510 | 0.160 | -3.180 | *** |
+-------------------------+-------------+--------+---------+--------------+
| ln population in range | -0.030 | 0.030 | -1.040 | |
+-------------------------+-------------+--------+---------+--------------+
| ln time to downtown | -0.250 | 0.100 | -2.590 | ** |
+-------------------------+-------------+--------+---------+--------------+
| income X average income | 0.370 | 0.090 | 4.010 | *** |
+-------------------------+-------------+--------+---------+--------------+
Estimating parameters for segment = (1, 2.0), size = 1922
Specifying model in 0.118133
average_price ln_average_income ln_population_in_range \
count 192200.000000 192200.000000 192200.000000
mean 12.183984 10.941919 11.869146
std 0.449140 1.206091 1.483376
min 0.000000 0.000000 0.000000
25% 11.997782 10.849898 11.672592
50% 12.191532 11.077796 12.385051
75% 12.396771 11.299900 12.808786
max 14.974602 11.819008 13.235408
ln_time_to_downtown income X average_income
count 192200.000000 192200.000000
mean 3.337886 21.881075
std 0.473631 2.354471
min 0.000000 0.000000
25% 3.029969 21.874495
50% 3.227324 22.142581
75% 3.750210 22.382541
max 4.637399 23.142420
[8 rows x 5 columns]
Null Log-liklihood: -8851.137097
Log-liklihood at convergence: -8302.212522
Log-liklihood ratio: 0.062017
+-------------------------+-------------+--------+---------+--------------+
| Variables | Coefficient | Stderr | T-score | Significance |
+=========================+=============+========+=========+==============+
| average price | -0.690 | 0.090 | -8.110 | *** |
+-------------------------+-------------+--------+---------+--------------+
| ln average income | 1.280 | 0.160 | 7.950 | *** |
+-------------------------+-------------+--------+---------+--------------+
| ln population in range | -0.030 | 0.030 | -0.880 | |
+-------------------------+-------------+--------+---------+--------------+
| ln time to downtown | -0.160 | 0.090 | -1.790 | * |
+-------------------------+-------------+--------+---------+--------------+
| income X average income | -0.560 | 0.090 | -6.280 | *** |
+-------------------------+-------------+--------+---------+--------------+
Estimating parameters for segment = (1, 3.0), size = 2232
Specifying model in 0.124567
average_price ln_average_income ln_population_in_range \
count 223200.000000 223200.000000 223200.000000
mean 12.180828 10.937054 11.863034
std 0.464468 1.232848 1.494587
min 0.000000 0.000000 0.000000
25% 11.997782 10.859368 11.662500
50% 12.191108 11.077796 12.385051
75% 12.396519 11.299900 12.808786
max 14.974602 11.819008 13.235408
ln_time_to_downtown income X average_income
count 223200.000000 223200.000000
mean 3.338770 22.602179
std 0.475647 2.511111
min 0.000000 0.000000
25% 3.027899 22.521670
50% 3.227871 22.826957
75% 3.750210 23.155910
max 4.562793 25.290298
[8 rows x 5 columns]
Null Log-liklihood: -10278.739855
Log-liklihood at convergence: -9343.506140
Log-liklihood ratio: 0.090987
+-------------------------+-------------+--------+---------+--------------+
| Variables | Coefficient | Stderr | T-score | Significance |
+=========================+=============+========+=========+==============+
| average price | -0.600 | 0.090 | -6.780 | *** |
+-------------------------+-------------+--------+---------+--------------+
| ln average income | 3 | 0.120 | 24.040 | *** |
+-------------------------+-------------+--------+---------+--------------+
| ln population in range | -0.070 | 0.030 | -2.740 | ** |
+-------------------------+-------------+--------+---------+--------------+
| ln time to downtown | -0.330 | 0.090 | -3.870 | *** |
+-------------------------+-------------+--------+---------+--------------+
| income X average income | -1.370 | 0.050 | -27.040 | *** |
+-------------------------+-------------+--------+---------+--------------+
Estimating parameters for segment = (2, 0.0), size = 1364
Specifying model in 0.079543
average_price ln_average_income ln_population_in_range \
count 136400.000000 136400.000000 136400.000000
mean 12.183381 10.940120 11.876167
std 0.436232 1.201284 1.477286
min 0.000000 0.000000 0.000000
25% 11.997782 10.843286 11.691214
50% 12.191078 11.072707 12.392602
75% 12.396519 11.294388 12.812449
max 14.974602 11.819008 13.235408
ln_time_to_downtown income X average_income
count 136400.000000 136400.000000
mean 3.333650 18.600222
std 0.474327 5.758796
min 0.000000 0.000000
25% 3.016206 19.807667
50% 3.227324 20.453672
75% 3.745288 20.886516
max 4.476772 21.894430
[8 rows x 5 columns]
Null Log-liklihood: -6281.452134
Log-liklihood at convergence: -5552.542282
Log-liklihood ratio: 0.116042
+-------------------------+-------------+--------+---------+--------------+
| Variables | Coefficient | Stderr | T-score | Significance |
+=========================+=============+========+=========+==============+
| average price | -1.930 | 0.070 | -28.210 | *** |
+-------------------------+-------------+--------+---------+--------------+
| ln average income | -0.050 | 0.060 | -0.760 | |
+-------------------------+-------------+--------+---------+--------------+
| ln population in range | 0.200 | 0.040 | 5.200 | *** |
+-------------------------+-------------+--------+---------+--------------+
| ln time to downtown | -0.930 | 0.090 | -10.580 | *** |
+-------------------------+-------------+--------+---------+--------------+
| income X average income | 0.140 | 0.010 | 27.070 | *** |
+-------------------------+-------------+--------+---------+--------------+
Estimating parameters for segment = (2, 1.0), size = 885
Specifying model in 0.050293
average_price ln_average_income ln_population_in_range \
count 88500.000000 88500.000000 88500.000000
mean 12.180102 10.935971 11.872944
std 0.461557 1.216387 1.488052
min 0.000000 0.000000 0.000000
25% 11.991727 10.843286 11.672592
50% 12.191078 11.072011 12.394157
75% 12.386533 11.296932 12.813586
max 14.974602 11.819008 13.235408
ln_time_to_downtown income X average_income
count 88500.000000 88500.000000
mean 3.333473 21.247073
std 0.478473 2.312345
min 0.000000 0.000000
25% 3.011896 21.222132
50% 3.227233 21.502109
75% 3.750210 21.756595
max 4.562793 22.587577
[8 rows x 5 columns]
Null Log-liklihood: -4075.575615
Log-liklihood at convergence: -3581.895134
Log-liklihood ratio: 0.121131
+-------------------------+-------------+--------+---------+--------------+
| Variables | Coefficient | Stderr | T-score | Significance |
+=========================+=============+========+=========+==============+
| average price | -1.170 | 0.090 | -13.110 | *** |
+-------------------------+-------------+--------+---------+--------------+
| ln average income | -2.330 | 0.200 | -11.460 | *** |
+-------------------------+-------------+--------+---------+--------------+
| ln population in range | 0.400 | 0.060 | 6.470 | *** |
+-------------------------+-------------+--------+---------+--------------+
| ln time to downtown | -0.170 | 0.150 | -1.180 | |
+-------------------------+-------------+--------+---------+--------------+
| income X average income | 1.300 | 0.120 | 11.020 | *** |
+-------------------------+-------------+--------+---------+--------------+
Estimating parameters for segment = (2, 2.0), size = 572
Specifying model in 0.039890
average_price ln_average_income ln_population_in_range \
count 57200.000000 57200.000000 57200.000000
mean 12.176781 10.930878 11.863699
std 0.490314 1.242152 1.498532
min 0.000000 0.000000 0.000000
25% 11.991727 10.843286 11.672592
50% 12.191078 11.072707 12.385051
75% 12.387834 11.294388 12.812392
max 14.974602 11.819008 13.235408
ln_time_to_downtown income X average_income
count 57200.000000 57200.000000
mean 3.336446 21.832742
std 0.477683 2.425241
min 0.000000 0.000000
25% 3.027899 21.847138
50% 3.227324 22.106957
75% 3.747391 22.347178
max 4.637399 23.105387
[8 rows x 5 columns]
Null Log-liklihood: -2634.157346
Log-liklihood at convergence: -2360.475993
Log-liklihood ratio: 0.103897
+-------------------------+-------------+--------+---------+--------------+
| Variables | Coefficient | Stderr | T-score | Significance |
+=========================+=============+========+=========+==============+
| average price | -0.970 | 0.160 | -5.950 | *** |
+-------------------------+-------------+--------+---------+--------------+
| ln average income | -0.460 | 0.280 | -1.620 | . |
+-------------------------+-------------+--------+---------+--------------+
| ln population in range | 0.250 | 0.070 | 3.490 | *** |
+-------------------------+-------------+--------+---------+--------------+
| ln time to downtown | -0.260 | 0.180 | -1.470 | . |
+-------------------------+-------------+--------+---------+--------------+
| income X average income | 0.320 | 0.160 | 1.960 | * |
+-------------------------+-------------+--------+---------+--------------+
Estimating parameters for segment = (2, 3.0), size = 268
Specifying model in 0.020312
average_price ln_average_income ln_population_in_range \
count 26800.000000 26800.000000 26800.000000
mean 12.176987 10.933148 11.873304
std 0.499580 1.229631 1.486060
min 0.000000 0.000000 0.000000
25% 11.991727 10.843286 11.691214
50% 12.191078 11.074403 12.392602
75% 12.395797 11.291993 12.812449
max 14.974602 11.819008 13.235408
ln_time_to_downtown income X average_income
count 26800.000000 26800.000000
mean 3.333686 22.485526
std 0.479936 2.482985
min 0.000000 0.000000
25% 3.011768 22.441104
50% 3.225800 22.726573
75% 3.745288 23.013566
max 4.476772 24.446476
[8 rows x 5 columns]
Null Log-liklihood: -1234.185610
Log-liklihood at convergence: -1108.039338
Log-liklihood ratio: 0.102210
+-------------------------+-------------+--------+---------+--------------+
| Variables | Coefficient | Stderr | T-score | Significance |
+=========================+=============+========+=========+==============+
| average price | -0.560 | 0.210 | -2.610 | ** |
+-------------------------+-------------+--------+---------+--------------+
| ln average income | 0.970 | 0.330 | 2.900 | ** |
+-------------------------+-------------+--------+---------+--------------+
| ln population in range | 0 | 0.090 | 0.050 | |
+-------------------------+-------------+--------+---------+--------------+
| ln time to downtown | -0.820 | 0.220 | -3.690 | *** |
+-------------------------+-------------+--------+---------+--------------+
| income X average income | -0.400 | 0.160 | -2.470 | ** |
+-------------------------+-------------+--------+---------+--------------+
Finished executing in 6.068114 seconds
Fetching annual_household_relocation_rates
Fetching modify_table
Traceback (most recent call last):
File "run_json.py", line 12, in <module>
for arg in args: misc.run_model(arg,dset,estimate=1,simulate=1)
File "/Users/knaaptime/anaconda/lib/python2.7/site-packages/urbansim-0.1.0-py2.7.egg/synthicity/utils/misc.py", line 21, in run_model
model.simulate(dset,config,year,show=show,variables=variables)
File "/Users/knaaptime/anaconda/lib/python2.7/site-packages/urbansim-0.1.0-py2.7.egg/synthicity/urbansim/locationchoicemodel.py", line 82, in simulate
movers = dset.relocation_rates(choosers,rate_table,rate_field)
File "/Users/knaaptime/anaconda/lib/python2.7/site-packages/urbansim-0.1.0-py2.7.egg/synthicity/urbansim/dataset.py", line 146, in relocation_rates
agents.relocation_rate.values[np.prod(a,axis=0).astype('bool')] = row[rate_fname]
File "/Users/knaaptime/anaconda/lib/python2.7/site-packages/numpy/core/fromnumeric.py", line 2112, in prod
out=out, keepdims=keepdims)
File "/Users/knaaptime/anaconda/lib/python2.7/site-packages/numpy/core/_methods.py", line 22, in _prod
out=out, keepdims=keepdims)
File "/Users/knaaptime/anaconda/lib/python2.7/site-packages/pandas/core/ops.py", line 496, in wrapper
arr = na_op(lvalues, rvalues)
File "/Users/knaaptime/anaconda/lib/python2.7/site-packages/pandas/core/ops.py", line 443, in na_op
raise_on_error=True, **eval_kwargs)
File "/Users/knaaptime/anaconda/lib/python2.7/site-packages/pandas/computation/expressions.py", line 175, in evaluate
**eval_kwargs)
File "/Users/knaaptime/anaconda/lib/python2.7/site-packages/pandas/computation/expressions.py", line 103, in _evaluate_numexpr
**eval_kwargs)
File "/Users/knaaptime/anaconda/lib/python2.7/site-packages/numexpr/necompiler.py", line 739, in evaluate
NumExpr(ex, signature, **context)
File "/Users/knaaptime/anaconda/lib/python2.7/site-packages/numexpr/necompiler.py", line 555, in NumExpr
precompile(ex, signature, context)
File "/Users/knaaptime/anaconda/lib/python2.7/site-packages/numexpr/necompiler.py", line 498, in precompile
ast = typeCompileAst(ast)
File "/Users/knaaptime/anaconda/lib/python2.7/site-packages/numexpr/necompiler.py", line 163, in typeCompileAst
% (ast.value + '_' + retsig+basesig))
NotImplementedError: couldn't find matching opcode for 'mul_bbb'
Closing remaining open files: /Users/knaaptime/urbansim-foti/data/mrcog.h5... done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment