Metacritic Review Scrape

README.md

Scrape Reviews from Metacritic

1. Browse a metacritic page of results like: All-Time Nintendo DS Reviews
2. Open developer tools > Console
3. Paste the javascript into the console prompt
4. Execute the javascript
5. the exported json is now in your clipboard. paste the json into a file and save.

Notes:

• You can get 100 results per page so it isn't too bad doing it this way.
• My purposes were to export top reviews for discontinued video game platforms. This is a one time operation so I didn't need further automation.

html-to-json.js

let items = Array.from(document.querySelectorAll('li.game_product div.product_wrap')).map((d) => {
  let url = d.children[0].children[0].href
  let extraData = d.children[2].innerText
  let extraDataItems = extraData.split('\n')
  let userScore = parseFloat(extraDataItems[0].replace('User: ', ''))
  let releaseDate
  let hardwareCompat = 'none'
  
  if (extraDataItems.length === 3) {
    releaseDate = extraDataItems[2]
    hardwareCompat = extraDataItems[1]
  } else {
    releaseDate = extraDataItems[1]
  }

  return {
    platform: (new URL(url)).pathname.split('/')[2],
    title: d.children[0].innerText,
    hardware_compatibility: hardwareCompat,
    link: url,
    metascore: parseInt(d.children[1].innerText, 10),
    user_score: userScore,
    release_date: releaseDate,
  }
})

copy(items)