UPDATED 22.11.2022
It's been two years since the last update, so here's the updated working script as per the comments below.
Thanks to BryanHaley for this.
setInterval(function () {
video = document.getElementsByTagName('ytd-playlist-video-renderer')[0];
video.querySelector('#primary button[aria-label="Action menu"]').click();
var things = document.evaluate(
'//span[contains(text(),"Remove from")]',
document,
null,
XPathResult.ORDERED_NODE_SNAPSHOT_TYPE,
null
);
for (var i = 0; i < things.snapshotLength; i++)
{
things.snapshotItem(i).click();
}
}, 500);
Non-english users will need to change "Action menu" and "Remove from" to what YouTube uses for their localization.
It worked for some time but then started saying 'this operation cannot be performed'. Not only can I delete stuff now but I cannot even add stuff. Google really wants you to not clear watch later to keep you addicted.