Skip to content

Instantly share code, notes, and snippets.

@scrapehero
Last active February 25, 2022 23:52
Show Gist options
  • Save scrapehero/cefaf014076b953f865a63ad453d507b to your computer and use it in GitHub Desktop.
Save scrapehero/cefaf014076b953f865a63ad453d507b to your computer and use it in GitHub Desktop.
Sitemap to extract data of review listings of a single product on Amazon.com using Web Scraper Chrome Extension. Instructions - https://www.scrapehero.com/amazon-review-scraper/
{
"_id": "amazon_reviews",
"startUrl": [
"https://www.amazon.com/Screen-Protector-SPARIN-Tempered-Glass/product-reviews/B013JZCAZK/ref=cm_cr_dp_d_show_all_btm?ie=UTF8&reviewerType=all_reviews"
],
"selectors": [
{
"id": "review",
"type": "SelectorElement",
"parentSelectors": [
"_root",
"next"
],
"selector": "div.a-section.review",
"multiple": true,
"delay": 0
},
{
"id": "author",
"type": "SelectorText",
"parentSelectors": [
"review"
],
"selector": "span.a-profile-name",
"multiple": false,
"regex": "",
"delay": 0
},
{
"id": "title",
"type": "SelectorText",
"parentSelectors": [
"review"
],
"selector": "a.a-size-base.review-title",
"multiple": false,
"regex": "",
"delay": 0
},
{
"id": "date",
"type": "SelectorText",
"parentSelectors": [
"review"
],
"selector": "span.a-size-base.a-color-secondary",
"multiple": false,
"regex": "",
"delay": 0
},
{
"id": "content",
"type": "SelectorText",
"parentSelectors": [
"review"
],
"selector": "div.a-row.review-data span.a-size-base",
"multiple": false,
"regex": "",
"delay": 0
},
{
"id": "rating",
"type": "SelectorText",
"parentSelectors": [
"review"
],
"selector": "span.a-icon-alt",
"multiple": false,
"regex": "",
"delay": 0
},
{
"id": "next",
"type": "SelectorLink",
"parentSelectors": [
"_root",
"next"
],
"selector": "li.a-last a",
"multiple": false,
"delay": 0
}
]
}
@stevencys
Copy link

Hi. I am a newbie here. This is an awesome tool. I would like to get some help from the community. I am using the JSON file provided here.

I would like to edit the JSON file to do 3 things:

  1. Add the product name as a column in each line in the output. For eg: New iPad 9.7" (2018 & 2017)...
  2. Remove the next and next-href columns in the output
  3. Remove the web-scraper-start-url column in the output

Can someone please share the code to be able to do the above?

Many thanks.
Steven

@LeeMeng2020
Copy link

This sitemap is a bit old (Jan 2019) and it no longer works in 2020. I have forked the project, fixed it, and also improved on it. Check out the Forks section, or my GitHub gists.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment