Project Notes 7/9/2012

notes

I got the crazy idea that the project could be released as a Puppy / DSL linux or #! Linux distro
  A debian or ubuntu compatible distro would be ideal, a tiny linux is very seductive.

    1) Code Updates could be pushed to the nodes
    2) A compute engine can be installed and used for grid operations (There goes the dream of tiny)
       MOsix, OpenMosix, Rocks are all compute engines
       SaltStack might be ideal, hacking a new setup with zeromq might be worth the effort.
    3) Volunteers can run the distro in a vm maintaining their own computers and volunteering compute time.

I spent today looking into GIS shapefile utils
  City of San Antonio provied shapefiles of the city
  State of texas proveds shape files of Texas
  Open Street Map and Open Layers provide shape files
  There is a proliferation of javascript libraries for mapping
  Shapefiles can be converted to json

I did some more searching for earthquake prediction papers
  Most valuable reading materials are anything specifically about spatial temporal statistics.
  Now need to filter available materials for relevance and approachability
  I am sure I have ~200 papers to sort through

The munging script can be broken into the following packages
    munge/
        xpdf
        db
        fileIO



regarding cleaning of data

1) dates are not in a uniform format
   most dates are input as m/d/yy
   some are m/d/yyyy
   >>> set([u'01/2012',
     u'11/2011',
     u'3/2012',
     u'5/2012',
     u'8/2011',
     u'2/2012',
     u'10/2011'])

2) there seems to be 88 unique call types but many overlap
   >>>set([u'',
     u'fire onlygrass',
     u'traffic stop',
     u'suspicious vehicle',
     u'prowler',
     u'ordinance violation',
     u'swat',
     u'burglary vehicle in progress',
     u'accident major officer ^',
     u'criminal mischief',
     u'suspicious person',
     u'sapd non-emergency call',
     u'internet predator',
     u'cutting',
     u'theft',
     u'burglary',
     u'missing person',
     u'shoplifting in progress',
     u'narcotic laws',
     u'robbery',
     u'suspicious vehicle with gun',
     u'theft in progress',
     u'robbery in progress',
     u'high water',
     u'fire only-grass ^',
     u'sapd nonemergency call',
     u'weapons',
     u'high water rescue',
     u'fire onlygrass ^',
     u'lewd conduct',
     u'wanted person',
     u'burglary in progress',
     u'abandoned vehicle',
     u'*sapd call nonemerg ^',
     u'accident major',
     u'elevator alarm',
     u'id theft',
     u'drowning',
     u'child locked in vehicle',
     u'holdup alarm in progress ^',
     u'sapd emergency call',
     u'threat - bomb with device',
     u'officer in trouble in progress',
     u'shoplifting',
     u'officer traffic stop',
     u'cutting in progress',
     u'suspicious person with gun',
     u'property found',
     u'officer on site activity',
     u'public lewdness',
     u'miscellaneous in progress ^',
     u'shooting in progress',
     u'assist the public',
     u'property lost',
     u'forgery',
     u'wrong way driver',
     u'missing person/runaway',
     u'traffic violation',
     u'yassist outside agency ^',
     u'missing person endangered',
     u'recovered stolen property',
     u'fire',
     u'traffic related',
     u'liquor law violation',
     u'theft of vehicle in progress',
     u'robbery of individual',
     u'threat bomb with device',
     u'accident minor',
     u'holdup alarm in progress',
     u'theft of vehicle',
     u'*sapd call emerg ^',
     u'burglary vehicle',
     u'shooting',
     u'information',
     u'suspicious person with knife',
     u'bomb with device',
     u'criminal mischief in progress',
     u'dwi',
     u'missing child',
     u'missing person - endangered',
     u'miscellaneous',
     u'animal related',
     u'threats',
     u'accident private property',
     u'robbery of individual progress',
     u'vice',
     u'accident minor officer ^',
     u'forgery in progress'])