Skip to content

Instantly share code, notes, and snippets.

@dkapitan
Last active December 30, 2016 12:33
Show Gist options
  • Save dkapitan/5f02c34a68ac59d0b877c3ad0b669b91 to your computer and use it in GitHub Desktop.
Save dkapitan/5f02c34a68ac59d0b877c3ad0b669b91 to your computer and use it in GitHub Desktop.
Fast date conversion with pandas.to_datetime
def date_lookup(s):
"""
http://stackoverflow.com/questions/29882573/pandas-slow-date-conversion
This is an extremely fast approach to datetime parsing.
For large data, the same dates are often repeated. Rather than
re-parse these, we store all unique dates, parse them, and
use a lookup to convert all dates.
"""
dates = {date:pd.to_datetime(date, infer_datetime_formt=True) for date in s.unique()}
return s.map(dates)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment