Skip to content

Instantly share code, notes, and snippets.

@StephenOTT
Created September 5, 2013 05:07
Show Gist options
  • Save StephenOTT/6446293 to your computer and use it in GitHub Desktop.
Save StephenOTT/6446293 to your computer and use it in GitHub Desktop.
Old code for converting date formats from XML format into proper format for Mongodb - Originally built for use with City of Ottawa Health Inspection data but was dropped in favour of xpath and conversion during variable creation
def convertDatesForMongo(parsedXML)
# Fixes Date Strings in Facility/Restarant information
# If statement is used to ensure that the date is not null otherway the strptime would throw a exception if it was null
# If Statement is only used because of data inconsistancies with Health Inspection Data
if parsedXML["doc"]["str"]["fs_fcr_date"] != nil
parsedXML["doc"]["str"]["fs_fcr_date"] = DateTime.strptime(parsedXML["doc"]["str"]["fs_fcr_date"][0..-5], '%Y-%m-%d %H:%M:%S').to_time.utc
end
if parsedXML["doc"]["str"]["fs_fefd"] != nil
parsedXML["doc"]["str"]["fs_fefd"] = DateTime.strptime(parsedXML["doc"]["str"]["fs_fefd"][0..-5], '%Y-%m-%d %H:%M:%S').to_time.utc
end
if parsedXML["doc"]["str"]["fs_fstlu"] != nil
parsedXML["doc"]["str"]["fs_fstlu"] = DateTime.strptime(parsedXML["doc"]["str"]["fs_fstlu"][0..-5], '%Y-%m-%d %H:%M:%S').to_time.utc
end
# Fixes all English Inspection date strings
# First if statement checks to see if there are any inspections that need to be modified. This is done by checking to see if the fs_insp_en hash in empty/null
if parsedXML["doc"]["arr"]["fs_insp_en"] != nil
#OLD CODE
# parsedXML["doc"]["arr"]["fs_insp_en"]["inspection"].update(parsedXML["doc"]["arr"]["fs_insp_en"]["inspection"]) do |key, oldValue, newValue|
# if key == 'inspectiondate' and oldValue != nil
# newValue = DateTime.strptime(oldValue[0..-5], '%Y-%m-%d %H:%M:%S').to_time.utc
# else
# oldValue
# end
# end
puts "DOGGGGGG: " + parsedXML["doc"]["arr"]["fs_insp_en"]["inspection"].key?("inspectiondate").to_s
parsedXML["doc"]["arr"]["fs_insp_en"]["inspection"].each do |key, value|
if key["inspectiondate"] != nil
parsedXML["doc"]["arr"]["fs_insp_en"]["inspection"][key] = DateTime.strptime(value[0..-5], '%Y-%m-%d %H:%M:%S').to_time.utc
puts key
puts value
end
if key["closuredate"] != nil
value = DateTime.strptime(value[0..-5], '%Y-%m-%d %H:%M:%S').to_time.utc
end
end
end
puts pp parsedXML
# Fixes all French Inspection date string
# First if statement checks to see if there are any inspections that need to be modified. This is done by checking to see if the fs_insp_fr hash in empty/null
#if parsedXML["doc"]["arr"]["fs_insp_fr"] != nil
# parsedXML["doc"]["arr"]["fs_insp_fr"]["inspection"].each do |key, value|
# if key["inspectiondate"] != nil
# key["inspectiondate"] = DateTime.strptime(key["inspectiondate"][0..-5], '%Y-%m-%d %H:%M:%S').to_time.utc
# end
# if key["closuredate"] != nil
# key["closuredate"] = DateTime.strptime(key["closuredate"][0..-5], '%Y-%m-%d %H:%M:%S').to_time.utc
# end
# end
#end
# Send fixed parsedXML into MongoDB
self.putHealthXMLinMongo(parsedXML)
end
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment