Skip to content

Instantly share code, notes, and snippets.

@dumplingcompromise
Last active February 20, 2021 22:09
Show Gist options
  • Save dumplingcompromise/7354080af12b58c09d685f50876e365f to your computer and use it in GitHub Desktop.
Save dumplingcompromise/7354080af12b58c09d685f50876e365f to your computer and use it in GitHub Desktop.
decentraland api looping
df = pd.DataFrame()
#update parameter used in mystring to start querying the database at the earliest update date of sale. The update
#date is specified in epoch date and needs to be converted to datetime for human consumption.
update = 1
while True:
#query the data using GraphQL python library.
query = gql(mystring.format(update))
result = client.execute(query)
#if there is no data returned it means you reached the end and should stop querying.
if len(client.execute(query)['orders']) <= 1:
break
else:
#Create a temporary dataframe to later append to the final dataframe that compiles all 1000-row dataframes.
df1 = pd.DataFrame()
df1 = pd.DataFrame(result['orders'])
#unfold a subdict into a series of columns.
df1 = df1.join(df1['nft'].apply(pd.Series))
#append your temp dataframe to your master dataset.
df = df.append(df1)
#Pass into the API the max date from your dataset to fetch the next 1000 records.
update = df['updatedAt'].max()
#reformat the update date in human-readable datetime format.
df['updatedAt_dt'] = df['updatedAt'].apply(lambda x: time.strftime('%Y-%m-%d', time.localtime(int(x))) )
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment