Batch download Google Groups attachments

google_groups_batch_download.md

A previous known tool to batch download Google Groups messages, https://github.com/icy/google-group-crawler, does not work with the current (as of March 2023) Google Groups website.

What I needed was to download all attachments from a small number of conversations. So I used some JavaScript to automate downloading them from my browser. It works like this:

• Open the Google Groups app and insert the initialization code into the Console:

function sleep(ms) {
  return new Promise(r => setTimeout(r, ms))
}

class Crawler {
  constructor() { this.links = new Set() }
  harvest() {
    Array.from(document.querySelectorAll('[aria-label=Download] a'))
      .map((a) => a.href)
      .forEach(i => this.links.add(i))
  }
  async openAll(interval) {
    this.stop = false
    for (const url of this.links) {
      window.open(url)
      await sleep(interval)
    }
  }
}

crawler = new Crawler()

• For each conversation that you are interested in: open the conversation, scroll all the way down to ensure all messages have been loaded, and run crawler.harvest() in the Console.

• Configure automatic download in your browser: set the appropriate download folder and enable the "save without asking" feature.

• Run crawler.openAll(2000) to start opening all links. 2000 is the interval between links (in milliseconds) and you can adjust it as you like.