jam53/CalculatingYouTubeWatchTime.md

## CalculatingYouTubeWatchTime.md

      
    Raw
  

              CalculatingYouTubeWatchTime.md
            
          
    Calculating YouTube watch time

Prerequisites

YouTube watch history

Request a copy of your YouTube watch history over at Google Takeout

Click on Deselect all
Scroll down and select YouTube and YouTube Music
Click on All YouTube data included
Make sure to only select history
Press OK
Click on Multiple formats and next to history pick JSON
Press OK

Click on Next step, leave the next screen as is and click on Create export.

You will receive an email shortly letting you know that your Google data is ready to download.

YouTube Data API v3 key


Navigate to the Google Cloud Platform
Press on Select a project in the left upper corner of the screen
In the newly opened window, click on NEW PROJECT
Leave the Project name and Location fields as is
Click on CREATE
Once the project is created, click on SELECT PROJECT from the notifications window
Click on APIs & Services towards the left side of the screen
Just below the search bar, click on + ENABLE APIS AND SERVICES
Inside the Search for APIs & Services search bar, enter youtube data api v3
Select the YouTube Data API v3 search result
Press ENABLE
Once enabled, click on CREATE CREDENTIALS towards the right side of the screen
Select Public data and press NEXT
Copy the API Key


Take note of this API Key, we will need it later


Press DONE

NodeJS

Download and install NodeJS
Required script


Create a file called CalculateYTWatchtime.js and copy-paste the code below into it.  
A quick and easy way to do this is by creating a text file, pasting the code, saving, and then renaming the file from CalculateYTWatchtime.txt to CalculateYTWatchtime.js. (To view the code, click on the triangle at the beginning of this line.)

//CalculateYTWatchtime.js
const fs = require("fs");

/**
 * Rounds a number to the specified number of decimal places.
 *
 * @param {number} number - The number to be rounded.
 * @param {number} decimals - The desired number of decimal places.
 * @returns {number} - The rounded number.
 */
function round(number, decimals)
{
    return Math.round((number + Number.EPSILON) * (10**decimals)) / (10**decimals)
}

/**
 * Converts a duration in seconds to a string representation of days, hours, minutes, and seconds.
 *
 * @param {number} seconds - The duration in seconds to be converted.
 * @returns {string} - The string representation of the duration in days, hours, minutes, and seconds.
 */
function secondsToDhms(seconds) 
{
    seconds = Number(seconds);

    const d = Math.floor(seconds / (3600*24));
    const h = Math.floor(seconds % (3600*24) / 3600);
    const m = Math.floor(seconds % 3600 / 60);
    const s = Math.floor(seconds % 60);
    
    const dDisplay = d > 0 ? d + (d == 1 ? " day, " : " days, ") : "";
    const hDisplay = h > 0 ? h + (h == 1 ? " hour, " : " hours, ") : "";
    const mDisplay = m > 0 ? m + (m == 1 ? " minute, " : " minutes, ") : "";
    const sDisplay = s > 0 ? s + (s == 1 ? " second" : " seconds") : "";

    return dDisplay + hDisplay + mDisplay + sDisplay;
}

/**
 * Converts an ISO 8601 duration to the corresponding duration in seconds.
 *
 * @param {string} duration - The ISO 8601 duration string to be converted.
 * @returns {number} - The duration in seconds.
 */
function convertISO8601DurationToSeconds(duration) 
{
    //Duration zero || duration more than a day (lofi girl stream e.g. or super similar super long videos)
    if (duration === "P0D" || duration.includes("D"))
    {
        return 0;
    }

    const match = duration.match(/PT(\d+H)?(\d+M)?(\d+S)?/);

    const hours = (parseInt(match[1]) || 0);
    const minutes = (parseInt(match[2]) || 0);
    const seconds = (parseInt(match[3]) || 0);

    const totalSeconds = (hours*60*60) + (minutes*60) + seconds;

    return totalSeconds <= maxVideoDurationSeconds ? totalSeconds : 0; //purge videos longer than `maxVideoDurationSeconds` seconds. These are often streams (or super long videos) and such that we never fully watched.
}

/**
 * Retrieves the durations of videos corresponding to the given video IDs.
 *
 * @param {string[]} videoIds - An array of video IDs for which to retrieve the durations.
 * @returns {Promise<Object[]>} - A promise that resolves to an array of objects containing the video ID and duration in seconds.
 */
async function getVideoDurations(videoIds) 
{
    let queriedData = []; //Contains an array of objects with the id and durationSeconds of each video
    let idsToQuery = []; 

    const queryIds = async (ids) => {
        console.log(`Querying progress: ${round(((queriedData.length + ids.length)/videoIds.length)*100, 2)}% - ${queriedData.length + ids.length}/${videoIds.length}`);
        const response = await fetch(`https://www.googleapis.com/youtube/v3/videos?id=${ids.join(',')}&key=${ytApiKey}&part=contentDetails`);
        const data = await response.json();
        data.items.forEach(item => {
            const duration = item.contentDetails.duration;
            const formattedDuration = convertISO8601DurationToSeconds(duration);
            queriedData.push({ id: item.id, durationSeconds: formattedDuration });
        });
    };

    for (id of videoIds)
    {
        if (idsToQuery.length < 50) //The maximum number of videos we can request from the API is 50
        {
            idsToQuery.push(id);
        }
        else
        {
            await queryIds(idsToQuery);
            idsToQuery = [];
        }
    }

    if (idsToQuery.length > 0) 
    {
        await queryIds(idsToQuery);
    }

    return queriedData;
}


/**
 * Entry point that retrieves watch history data, processes it, and calculates the total watch time.
 */
const watchHistoryFileName = process.argv[2]; //The JSON file containing the watch history data
const ytApiKey = process.argv[3]; //YouTube Data API v3 key
const maxVideoDurationSeconds = process.argv[4] || 2400; //Videos longer than this duration wont be taken into consideration when calculating the total watch time

(async() => {
const data = fs.readFileSync(watchHistoryFileName, 'utf8');

// Parse the JSON data
try 
{
    const watchHistory = JSON.parse(data);

    const stillAvailableVideos = watchHistory.map(element => {
        if (element.titleUrl != null)
        {
            const youtubeUrl = element.titleUrl;
            const indexId = youtubeUrl.indexOf("watch?v=") + "watch?v=".length;
            const videoId = youtubeUrl.substring(indexId);

            return {id: videoId, published: element.time, title: element.title, channel: element.subtitles ? element.subtitles[0].name : "Unknown"};
        }
        else //When a video has been deleted
        {
            return null;
        }      
    }).filter(element => element); //Remove the `null` elements, i.e. the deleted videos

    let availableVideosWithWatchtime = await getVideoDurations(stillAvailableVideos.map(video =>  video.id));

    let merged = [];
    for(let i=0; i < availableVideosWithWatchtime.length; i++) 
    {
        console.log(`Merging progress: ${round((i/availableVideosWithWatchtime.length)*100, 2)}% - ${i}/${availableVideosWithWatchtime.length}`);

        merged.push({
         ...availableVideosWithWatchtime[i], 
         ...(stillAvailableVideos.find((itmInner) => itmInner.id === availableVideosWithWatchtime[i].id))}
        ); //Combine the 2 arrays into a single array, where each object in the merged array contains: id, durationSeconds, published, title and channel
    }

    fs.writeFileSync("./WatchHistoryWithDuration.json", JSON.stringify(merged, null, 4), "utf-8");

    console.log(`From: ${merged[merged.length - 1].published} until ${merged[0].published}`);
    console.log(`Total amount of videos: ${merged.length}`);
    console.log(`Total watchtime: ${secondsToDhms(merged.reduce((accumulator, video) => accumulator + video.durationSeconds, 0))}`)
} 
catch (parseError) 
{
    console.error('Error parsing JSON:', parseError);
}
})();

Using the tool

With all of the steps above completed, we are now ready to start using the tool.
Start by unzipping the Google Takeout zip file that you exported from Google Takeout. Afterwards, navigate to the subfolder containing the watch-history.json file and place the CalculateYTWatchtime.js file that you just created in the same folder as the watch-history.json file. Finally open a new terminal window in the folder that contains both of the previously mentioned files.

In Windows this can be done rather easily by following these steps:

Open File Explorer and open the folder containing both the watch-history.json and CalculateYTWatchtime.js files.
Click on the address bar in File Explorer.
Type "cmd" into the address bar.
Press Enter.


Once that's done, run the following command inside the terminal:
node CalculateYTWatchtime.js <name of the watch history file> <youtube api key>

Important note: Make sure to use the watch history json file, not the one related to the search history.
Example:
node CalculateYTWatchtime.js watch-history.json "oVqXyR9nMpFzLjQb4sHk6u2wGcVgI8hU1PdA3eT"


<name of the watch history file>

This should contain the path to the watch history file that was inside the zip.


<youtube data api v3 key>

The YouTube Data API v3 key that you previously generated.


Optionally, you can enter a third parameter maxVideoDurationSeconds to specify the maximum duration in seconds for videos to be considered in the total watch time calculation. Videos longer than this duration will be excluded. By default, the tool uses a value of 2400 seconds (40 minutes).
node CalculateYTWatchtime.js watch-history.json "oVqXyR9nMpFzLjQb4sHk6u2wGcVgI8hU1PdA3eT" 1800

This example sets the maxVideoDurationSeconds parameter to 1800 seconds (30 minutes).


Remember to replace the placeholder values with the actual file path and YouTube Data API key you obtained. After executing the command, the tool will process the watch history file, retrieve video durations using the YouTube Data API, and calculate the total watch time based on the videos available and their durations.
Furthermore, the tool will generate an output file named WatchHistoryWithDuration.json, which will contain the merged data including video details and durations. Additionally, the tool will display the start and end dates of the watch history, the total number of videos, and the total watch time in a human-readable format on the console.
Feel free to adjust the maxVideoDurationSeconds parameter as needed to customize the exclusion of longer videos from the watch time calculation.