Skip to content

Instantly share code, notes, and snippets.

@Dansyuqri
Created July 13, 2019 05:56
Show Gist options
  • Save Dansyuqri/82715d51b9ff181f73d041faa94a015f to your computer and use it in GitHub Desktop.
Save Dansyuqri/82715d51b9ff181f73d041faa94a015f to your computer and use it in GitHub Desktop.
Faster row retrieval using pandas

In pandas it is much faster to query a row independently of the headers and then using dict and zip to convert into header to column value mapping rather than using the in-built to_dict method.

import pandas as pd
import os, time

dataframe = pd.read_csv("data.csv", index_col=False)
start_time = time.time()
row = dataframe.loc[[25], list(dataframe)].to_dict("records")[0]
print("TIME: {}".format(time.time() - start_time))
>> TIME: 0.10403084754943848

start_time = time.time()
row = dataframe.loc[25, :]
row_data = dict(zip(list(dataframe), row))
print("TIME: {}".format(time.time() - start_time))
>> TIME: 0.00045490264892578125
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment