Last active
September 16, 2021 16:37
-
-
Save adidonato/5ebe2130b5287482a8aa584981f0ae9f to your computer and use it in GitHub Desktop.
Turn JSON object from Dune gql API into a CSV file
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import json | |
import csv | |
import sys | |
def main(): | |
filename = sys.argv[1] | |
with open(filename) as f: | |
data = json.load(f) | |
if data.get("data") is not None: | |
data = data.get("data") | |
with open("clean-{}.csv".format(filename.split(".")[0]), "w") as csv_file: | |
writer = csv.writer(csv_file) | |
keys = data.get("get_result_by_job_id", data.get("get_result_by_result_id"))[1].get("data").keys() | |
dict_writer = csv.DictWriter(csv_file, keys) | |
dict_writer.writeheader() | |
for obj in data.items(): | |
for objects in obj[1]: | |
actual_shit_you_want = objects.get("data") | |
if actual_shit_you_want is not None: | |
dict_writer.writerows([actual_shit_you_want]) | |
if __name__ == "__main__": | |
main() |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Get Dune data in JSON
graphql
requests, look at the ones with larger sizes (one will be history and another the resultset), pink box is what we're afterrugged-dune-data.json
clean-
plus the name of the original file in.csv
Example Output:
Tested on several queries including this one