Skip to content

Instantly share code, notes, and snippets.

@simonw
Created March 26, 2023 16:07
Show Gist options
  • Star 16 You must be signed in to star a gist
  • Fork 2 You must be signed in to fork a gist
  • Save simonw/8fbade64a3606645243b3438930b8159 to your computer and use it in GitHub Desktop.
Save simonw/8fbade64a3606645243b3438930b8159 to your computer and use it in GitHub Desktop.

Fetch and Save JSON

2023-03-26 01:44:42

user: Write a new JavaScript fetch() function which calls the old one but also saves the JSON that is retrieved - if it is used to retrieve JSON - in a window.fetchedData object


assistant: To achieve this, you can create a new function called fetchAndSaveJson that wraps the native fetch function. This new function will call the original fetch function, and if the response contains JSON data, it will save the JSON data to a window.fetchedData object. Here's how you can implement this:

// Define a global object to store fetched JSON data
window.fetchedData = {};

// Define a new fetch function that wraps the native fetch
async function fetchAndSaveJson(url, options) {
  try {
    // Call the native fetch function
    const response = await fetch(url, options);

    // Check if the response has a JSON content type
    const contentType = response.headers.get('content-type');
    if (contentType && contentType.includes('application/json')) {
      // If the response is JSON, parse it and save it to the fetchedData object
      const jsonData = await response.json();
      window.fetchedData[url] = jsonData;
      return jsonData;
    }

    // If the response is not JSON, return the original response
    return response;
  } catch (error) {
    // Handle any errors that occur during the fetch
    console.error('Error fetching and saving JSON:', error);
    throw error;
  }
}

// Example usage
fetchAndSaveJson('https://jsonplaceholder.typicode.com/todos/1')
  .then(data => {
    console.log('Fetched JSON data:', data);
    console.log('Data saved in window.fetchedData:', window.fetchedData);
  })
  .catch(error => {
    console.error('Error:', error);
  });

In this implementation, the fetchAndSaveJson function takes the same arguments as the native fetch function (url and options). It then calls the native fetch function with these arguments and checks the content-type header of the response to determine if the response contains JSON data. If the response is JSON, the function parses the JSON data, saves it to the window.fetchedData object using the URL as the key, and returns the parsed JSON data. If the response is not JSON, the function returns the original response.

Note that this implementation assumes that the window.fetchedData object is used exclusively for storing fetched JSON data, and that the URLs used to fetch data are unique. If there are multiple requests to the same URL, the data from the latest request will overwrite the data from previous requests.


user: No I need to replace the old window.fetch() function entirely - I want to patch it so that I can intercept calls made by other existing code


assistant: To replace the native window.fetch function and intercept calls made by other existing code, you can create a new function that wraps the original fetch function. You can then assign this new function to window.fetch, effectively "patching" the native fetch function. Here's how you can do this:

// Define a global object to store fetched JSON data
window.fetchedData = {};

// Store a reference to the original fetch function
const originalFetch = window.fetch;

// Define a new fetch function that wraps the original fetch
window.fetch = async function(url, options) {
  try {
    // Call the original fetch function
    const response = await originalFetch(url, options);

    // Check if the response has a JSON content type
    const contentType = response.headers.get('content-type');
    if (contentType && contentType.includes('application/json')) {
      // If the response is JSON, clone the response so we can read it twice
      const responseClone = response.clone();
      // Parse the JSON data and save it to the fetchedData object
      const jsonData = await responseClone.json();
      window.fetchedData[url] = jsonData;
    }

    // Return the original response
    return response;
  } catch (error) {
    // Handle any errors that occur during the fetch
    console.error('Error fetching and saving JSON:', error);
    throw error;
  }
};

// Example usage
fetch('https://jsonplaceholder.typicode.com/todos/1')
  .then(response => response.json())
  .then(data => {
    console.log('Fetched JSON data:', data);
    console.log('Data saved in window.fetchedData:', window.fetchedData);
  })
  .catch(error => {
    console.error('Error:', error);
  });

In this implementation, we store a reference to the original fetch function in the originalFetch variable. We then define a new function that calls originalFetch with the provided arguments, checks the content-type header of the response to determine if the response contains JSON data, and saves the JSON data to the window.fetchedData object using the URL as the key. Finally, we assign this new function to window.fetch, effectively replacing the native fetch function.

Note that we use response.clone() to create a clone of the response before reading the JSON data. This is necessary because the response body can only be read once, and we want to allow the caller to read the response body as well.

This implementation assumes that the window.fetchedData object is used exclusively for storing fetched JSON data, and that the URLs used to fetch data are unique. If there are multiple requests to the same URL, the data from the latest request will overwrite the data from previous requests.


@tedmiston
Copy link

I wanted to understand what some use cases would be for this, so I asked ChatGPT. I'm curious if its explanations were correct?

@lauriestark The author explains this himself in the blog post at https://simonwillison.net/2023/Mar/27/ai-enhanced-development/. The main use case is to create a system to archive conversations from ChatGPT.

But this post / gist were also created 2 months ago. IMO today it's easier to just use ChatGPT's built-in data export feature, although it is not real-time. In my experience, it's also oddly variable: sometimes the exports arrive in < 12 hours and sometimes they take nearly a week. No idea why that is; in the grand scheme of things it's a very tiny amount of data.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment