Skip to content

Instantly share code, notes, and snippets.

@p0n1
Last active July 12, 2023 11:39
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save p0n1/3c9554376b748908f604d1a3dd74fff4 to your computer and use it in GitHub Desktop.
Save p0n1/3c9554376b748908f604d1a3dd74fff4 to your computer and use it in GitHub Desktop.
Export Query Results From Dune Analytics

This Gist provides a set of JavaScript snippets that can be executed directly in the browser console to export query results from Dune Analytics. It includes methods to extract data from the first page of results, paginate through subsequent pages, and download the data into a .json file.

The selector for query results row and next page button may change over time. Find the correct one at that time.

Instructions

Open your web browser console. These snippets have been tested on Google Chrome.

Copy the desired snippet from the Gist and paste it into the console.

Press Enter to execute the command.

Test to get results of the first page

let rows = document.querySelectorAll('tbody[role="rowgroup"] > tr');
let data = Array.from(rows).map(row => {
    let cells = row.querySelectorAll('td > div');
    let rowData = Array.from(cells).map(cell => cell.textContent);
    return rowData;
});

console.log(data);

Check if the results fit your need.

Click over next pages to get all results

let allData = [];
let stop = false;
let pages = 1;
let timeout = 2000;

function extractData() {
    let rows = document.querySelectorAll('tbody[role="rowgroup"] > tr');
    let data = Array.from(rows).map(row => {
        let cells = row.querySelectorAll('td > div');
        let rowData = Array.from(cells).map(cell => cell.textContent);
        return rowData;
    });
    allData.push(...data);
}

function goToNextPage() {
    if (stop) {
      	console.log("stopped manually")
        console.log(allData); // Log all data when stopping
        return;
    }

    let nextButton = document.querySelector('#results > div > div._____slug___results__LdsXL > div > div.visual_vizFooter__RdwS4 > ul > li:nth-child(6) > button');
    if (nextButton && !nextButton.disabled) {
      	pages++;
        nextButton.click();
      	console.log("click next page", pages)
        setTimeout(() => {
            extractData();
            goToNextPage();
        }, timeout);
    } else {
      	console.log("no next page found, stop!")
        console.log(allData); // Log all data when there are no more pages or the next button is not clickable
    }
}


// Start the process
extractData();
goToNextPage();

If you want to stop the process manually, set stop to true.

Download data into a json file

function downloadData(data) {
    const blob = new Blob([JSON.stringify(data)], { type: 'text/json' });
    const url = URL.createObjectURL(blob);

    const a = document.createElement('a');
    a.download = 'data.json';
    a.href = url;
    a.click();
}

// When you want to download the data, call the following function:
downloadData(allData);
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment