muhdiboy/lastfm-automated-remove-duplicates.md

## lastfm-automated-remove-duplicates.md

      
    Raw
  

              lastfm-automated-remove-duplicates.md
            
          
    Why would I need this?

Your scrobbler might have decided to scrobble every song hundreds of times, and you can't really remove those scrobbles efficiently. Or you might have accidentally installed multiple scrobbler extensions at the same time - wondering why multiple scrobbles appear for every song played at a time - and you want to clear them after finding the issue.
Using this script still doesn't necessarily make the process quick since Last.fm only displays a limited number of scrobbles that can be removed on each page of your library.
However unlike the implementation of @sk22 and its forks, this UserScript, which is derived from those scripts, is run once. The rest of the process is automated and the script will stop at the page you have set using the prompt.
Installation

Prerequisites

You will need some form of UserScript interpreter/injector plugin and a compatible browser.

For Google Chrome and Chromium-based browsers (e.g. Vivaldi, Opera or Brave), you need the Tampermonkey plugin. Just click on the link or search for it yourself on the Chrome Web Store and add the extension.
Currently not supported: For Firefox, you also need the Tampermonkey plugin (or alternatively Greasemonkey). Just click on one of the desired links or search for it yourself on the Firefox Browser Add-Ons page.

How to install the UserScript

Just click on the "Raw" button in the upper right-hand corner of the script file or by clicking here. Your extension should open an installation window where you confirm the installation of the UserScript.
If that is not working, try to add the script manually to your extension by copying and pasting it.
Using the script - step by step


Open your Last.fm account's library while being logged in (https://www.last.fm/user/_/library). Navigate to the page from which you want to start removing duplicates by clicking through the pages at the bottom of the page or by entering the page number in the URL (e.g. https://www.last.fm/user/_/library?page=123).

Alternatively, you can choose a specific date range instead and navigate to the last or desired page.


Manually reload the page if the button doesn't appear.
You have the option to change the behaviour of the comparison process. By unchecking the checkbox, you can make the script ignore timestamps and compare over n titles. You'll be prompted to enter a number after clicking the button.
Click the button and enter the page number at which you want the script to stop.

If you have unchecked the checkbox, you will be prompted to enter a number for how many tracks you want to compare.


Now enjoy the purge :)

You can continue with other tasks on your PC. Just leave the window active in the background. I advise you to run the script in a separate window while doing other work. Using the same browser in another window is also fine. Avoid activating another tab, though.
It takes approximately 5 minutes to go through 100 pages (in my own case).
If you want to stop the script, you can click the Cancel button on the left-hand side.


When it reaches the page you have set at the prompt, the script will automatically stop.

Updating the script

You can set up the script to auto-update in the settings of the UserScript. Open up the plugin settings and select the "LastFM automated duplicate scrobble deletion script." Then, go to the script settings in the upper left-hand corner, check the "Search for updates" checkbox and save.
Alternatively, you can also update it manually by following the installation instructions above or searching for updates in the plugin overview.
Possible future updates


Ability to differentiate within days, hours or minutes (implemented on a different version here: https://gist.github.com/olphil99/161cd24cba9db74fc945f3ebf167a54a by @olphil99, currently not working though)

Alternatively, since version 1.2.1, the script is able to only remove duplicates from the same timestamp.


Done


Adding a button on the last.fm history page to initiate the script instead of activating and deactivating the Userscript.

added in v1.3


Implementing a "stop at page x" dialogue window.

added in v1.3


Implementing a "check x tracks for duplicates" dialogue window instead of editing "num" in the script

added in v1.3


Eliminate the need to manually edit the script by incorporating a checkbox or another method for configuring options.

added in v1.4


Thank-yous

Based on https://gist.github.com/sk22/39cc280840f9d82df574c15d6eda6629#gistcomment-3046698
Thanks to previous contributors @CennoxX, @mattsson, @gms8994, @huw and @sk22
Thanks to new contributions from @Eiron
Feel free to post any problems, fixes, improvements and feedback. I will try to help in any form.
To anyone having more experience in Javascript or OpenUserJS/UserScript: your help is also needed to improve the script by implementing the above-mentioned possible future updates as I do not have the necessary skills to contribute.

  
## lastfm-automated-remove-duplicates.user.js
// ==UserScript==
// @name         LastFM automated duplicate scrobble deletion script
// @namespace    https://gist.github.com/muhdiboy
// @version      1.4.5
// @downloadURL  https://gist.github.com/muhdiboy/a293cbff355af750e3b8f45ec816d1f1/raw/lastfm-automated-remove-duplicates.user.js
// @updateURL    https://gist.github.com/muhdiboy/a293cbff355af750e3b8f45ec816d1f1/raw/lastfm-automated-remove-duplicates.user.js
// @description  Based on https://gist.github.com/sk22/39cc280840f9d82df574c15d6eda6629#gistcomment-3046698, thanks to previous contributors (@CennoxX, @mattsson, @gms8994, @huw and @sk22) and new contributors (@Eiron).
// @author       muhdiboy
// @match        https://www.last.fm/*user/*
// @icon         data:image/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==
// @grant        none
// ==/UserScript==

(function() {
    // Variables
    var runScript = localStorage.getItem('runScript') || 'false';
    var rmOnlyRealDup = localStorage.getItem('rmOnlyRealDup') !== 'false'; // Set checkbox to checked/true as default
    var num = localStorage.getItem('num') || "5";
    var stopAtPage = localStorage.getItem('stopAtPage') || "0";
    var currentPage = extractQueryParam('page', window.location.href) || 1;
    window.addEventListener('click', function () {
        currentPage = extractQueryParam('page', window.location.href) || 1;
    });

    // Functions
    var pageURLCheckTimer = setInterval (
        function () {
            // Check the URL at regular intervals to add or remove buttons
            if ( this.lastPathStr !== location.pathname || this.lastQueryStr !== location.search) {
                this.lastPathStr = location.pathname;
                this.lastQueryStr = location.search;
                if (shouldMatchPage()) createButtons(); else removeButtons();
            }
        }
        , 125
    );

    function replaceQueryParam(param, newval, search) {
        // Function to replace a query parameter in the URL
        var regex = new RegExp("([?;&])" + param + "[^&;]*[;&]?");
        var query = search.replace(regex, "$1").replace(/&$/, '');
        return (query.length > 2 ? query + "&" : "?") + (newval ? param + "=" + newval : '');
    }

    function extractQueryParam( name, url ) {
        // Function to extract a query parameter from the URL
        if (!url) url = window.location.href;
        name = name.replace(/[\[]/,"\\\[").replace(/[\]]/,"\\\]");
        var regexS = "[\\?&]"+name+"=([^&#]*)";
        var regex = new RegExp( regexS );
        var results = regex.exec( url );
        return results == null ? null : results[1];
    }

    function shouldMatchPage() {
        // Function to determine if the current page should be matched
        var url = window.location.href;
        var excludeUrlRegex = /^https:\/\/www\.last\.fm\/.*user\/.*\/library\/(albums|artists|tracks|music).*/;
        var matchUrlRegex = /^https:\/\/www\.last\.fm\/.*user\/.*\/library.*/;
        return matchUrlRegex.test(url) && !excludeUrlRegex.test(url);
    }

    function removeButtons() {
        // Function to remove all buttons from the page
        var cancelButton = document.getElementById('cancelButton');
        var buttonContainer = document.getElementById('buttonContainer');

        if (cancelButton) cancelButton.remove();
        if (buttonContainer) buttonContainer.remove();
    }

    // Buttons
    function createButtons() {
        var cancelButton = document.createElement('button');
        cancelButton.id = 'cancelButton';
        cancelButton.innerHTML = 'Cancel<br>Script';
        cancelButton.style.position = 'fixed';
        cancelButton.style.bottom = '50%';
        cancelButton.style.left = '0';
        cancelButton.style.zIndex = '9999';
        cancelButton.style.fontFamily = 'monospace';
        cancelButton.style.padding = '3px';
        cancelButton.style.border = 'dashed';
        if (runScript === 'true') {
            cancelButton.style.color = 'white';
            cancelButton.style.background = 'red';
            cancelButton.style.borderColor = 'firebrick';
            cancelButton.addEventListener('click', function() {
                localStorage.setItem('runScript', 'false');
                location.reload();
            });
        } else {
            cancelButton.style.color = 'gray';
            cancelButton.style.background = 'lightgray';
            cancelButton.style.borderColor = 'lightgray';
            cancelButton.style.cursor = 'default';
        }
        document.body.appendChild(cancelButton);

        // Create the main button to trigger the script
        var runButton = document.createElement('button');
        runButton.textContent = 'Remove Duplicates';
        runButton.style.position = 'relative';
        runButton.style.display = 'block';
        runButton.style.marginBottom = '5px';
        runButton.style.fontFamily = 'sans-serif';
        runButton.style.fontSize = 'initial';
        runButton.style.border = 'outset';
        runButton.style.borderColor = 'crimson';
        runButton.style.color = 'white';
        runButton.style.background = 'firebrick';
        runButton.style.padding = '5px';
        runButton.addEventListener('click', function () {
            var stopLoop = false;

            while (!stopLoop) {
                var inputStopAtPage = prompt("Stop at which page?", stopAtPage);
                if (inputStopAtPage === null) return;
                if (!/^\d+$/.test(inputStopAtPage)) { alert("Invalid input. Please enter a valid number."); continue; }
                if (inputStopAtPage > currentPage && !(currentPage === 0 && inputStopAtPage === 1)) { alert("Please enter a page less than or equal to your current page: " + currentPage); continue; }
                localStorage.setItem('stopAtPage', inputStopAtPage);
                stopLoop = true;
            }

            if (!rmOnlyRealDup) {
                stopLoop = false;
                while (!stopLoop) {
                    var inputNum = prompt("Enter the number of tracks to check for duplicates\n(between 2 and 50):", num);
                    if (inputNum === null) return;
                    if (!/^\d+$/.test(inputNum) || inputNum < 2 || inputNum > 50) { alert("Invalid input. Please enter a valid number between 2 and 50."); continue; }
                    localStorage.setItem('num', inputNum);
                    stopLoop = true;
                }
            }

            localStorage.setItem('runScript', 'true');
            location.reload();
        });

        // Create the checkbox element
        var rmOnlyRealDupCheckbox = document.createElement('input');
        rmOnlyRealDupCheckbox.textContent = 'Only compare real duplicates?';
        rmOnlyRealDupCheckbox.type = 'checkbox';
        rmOnlyRealDupCheckbox.id = 'rmOnlyRealDupCheckbox';
        rmOnlyRealDupCheckbox.style.position = 'relative';
        rmOnlyRealDupCheckbox.style.marginLeft = '10px';
        rmOnlyRealDupCheckbox.checked = rmOnlyRealDup;
        rmOnlyRealDupCheckbox.addEventListener('change', function () {
            localStorage.setItem('rmOnlyRealDup', rmOnlyRealDupCheckbox.checked);
        });

        // Create the label for the checkbox
        var rmOnlyRealDupCheckboxLabel = document.createElement('label');
        rmOnlyRealDupCheckboxLabel.style.position = 'relative';
        rmOnlyRealDupCheckboxLabel.style.marginLeft = '5px';
        rmOnlyRealDupCheckboxLabel.style.fontFamily = 'sans-serif';
        rmOnlyRealDupCheckboxLabel.style.fontSize = 'medium';
        rmOnlyRealDupCheckboxLabel.setAttribute('for', 'rmOnlyRealDupCheckbox');
        rmOnlyRealDupCheckboxLabel.textContent = 'Only compare real duplicates?';

        // Create a container element to hold the main button and checkbox
        var buttonContainer = document.createElement('div');
        buttonContainer.id = 'buttonContainer';
        buttonContainer.style.position = 'absolute';
        buttonContainer.style.top = '370px';
        buttonContainer.style.left = '50%';
        buttonContainer.style.transform = 'translate(-50%, -50%)';
        buttonContainer.appendChild(runButton);
        buttonContainer.appendChild(rmOnlyRealDupCheckboxLabel);
        buttonContainer.appendChild(rmOnlyRealDupCheckbox);
        const metaElement = document.querySelector('.content-top-has-nav');
        metaElement.insertAdjacentElement('afterend', buttonContainer);
    }

    // Automated duplicate scrobble deletion and navigation logic
    if(shouldMatchPage()) {
        window.addEventListener('load', function () {
            // Event listener for when the page has finished loading
            if (runScript === 'true') {
                var found = 0;
                var sections = Array.from(document.getElementsByTagName("tbody"));

                sections.forEach(function (section) {
                    // Loop through each section
                    var els = Array.from(section.rows);
                    var names = els.map(function (el) {
                        var nmEl = el.querySelector('.chartlist-name');
                        var artEl = el.querySelector('.chartlist-artist');
                        if (rmOnlyRealDup) {
                            var tstEl = el.querySelector('.chartlist-timestamp');
                            // Construct the name string including track name, artist and timestamp
                            return nmEl && artEl && tstEl && nmEl.textContent.replace(/\s+/g, ' ').trim()+':'+artEl.textContent.replace(/\s+/g, ' ').trim()+':'+tstEl.textContent.replace(/\s+/g, ' ').trim();
                        } else {
                            // Construct the name string including only track name and artist
                            return nmEl && artEl && nmEl.textContent.replace(/\s+/g, ' ').trim()+':'+artEl.textContent.replace(/\s+/g, ' ').trim();
                        }
                    });

                    names.forEach(function (name, i, names) {
                        // Loop through each name in the section
                        if (!names.slice(i + 1, i + 1 + parseInt(num)).includes(name)) return;
                        // Check if the current name has duplicates within the specified range
                        var delBtn = els[i].querySelector('[data-ajax-form-sets-state="deleted"]');
                        if (delBtn) { delBtn.click(); found++; };
                        // If a delete button is found, click it and increment the counter
                    });
                });

                if (found > 0) {
                    // If duplicates were found, reload the page every 5 seconds
                    setInterval(function() {
                        location.reload();
                    }, 5000);
                }

                // If the URL contains "delete", go back one page in history
                if (window.location.toString().includes("delete")) history.go(-1);

                if (currentPage <= stopAtPage) {
                    // Stop the Script and message user
                    localStorage.setItem('runScript', 'false');
                    alert("LastFM duplicate deletion complete.");
                    location.reload();
                }

                // Go up one page
                else window.location.href = window.location.pathname + replaceQueryParam('page', extractQueryParam('page', window.location.href) - 1, window.location.search);
            }
        });
    }
})();