Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save chriswhong/ff86b482273ce20cc8dc2391a6b30c4c to your computer and use it in GitHub Desktop.
Save chriswhong/ff86b482273ce20cc8dc2391a6b30c4c to your computer and use it in GitHub Desktop.
Pulling a Qri versioned dataset from the distributed web into a pandas dataframe (python)
# an example of pulling in a CSV string from a local qri repo,
# and parsing it into a dataframe with pandas.read_csv()
import pandas as pd
import subprocess
from io import StringIO
# this Qri class should eventually have all the methods we would need for reading,
# writing, pushing, pulling, committing, etc.
class Qri:
def dataframe(self, ref):
# execute the qri get command in a subprocess
# decode the bytes into str using .decode()
# use StringIO to
command="qri get body " + ref
csv = StringIO(subprocess.check_output(command, shell=True).decode("utf-8"))
return pd.read_csv(csv)
qri = Qri()
# use qri.dataframe to bring in a dataframe for the specified dataset reference
qdf = qri.dataframe("chriswhong-demo/nyc_bids")
print(qdf)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment