Last active
February 20, 2021 22:09
-
-
Save dumplingcompromise/7354080af12b58c09d685f50876e365f to your computer and use it in GitHub Desktop.
decentraland api looping
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
df = pd.DataFrame() | |
#update parameter used in mystring to start querying the database at the earliest update date of sale. The update | |
#date is specified in epoch date and needs to be converted to datetime for human consumption. | |
update = 1 | |
while True: | |
#query the data using GraphQL python library. | |
query = gql(mystring.format(update)) | |
result = client.execute(query) | |
#if there is no data returned it means you reached the end and should stop querying. | |
if len(client.execute(query)['orders']) <= 1: | |
break | |
else: | |
#Create a temporary dataframe to later append to the final dataframe that compiles all 1000-row dataframes. | |
df1 = pd.DataFrame() | |
df1 = pd.DataFrame(result['orders']) | |
#unfold a subdict into a series of columns. | |
df1 = df1.join(df1['nft'].apply(pd.Series)) | |
#append your temp dataframe to your master dataset. | |
df = df.append(df1) | |
#Pass into the API the max date from your dataset to fetch the next 1000 records. | |
update = df['updatedAt'].max() | |
#reformat the update date in human-readable datetime format. | |
df['updatedAt_dt'] = df['updatedAt'].apply(lambda x: time.strftime('%Y-%m-%d', time.localtime(int(x))) ) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment