Last active
May 28, 2020 02:10
-
-
Save LeeMeng2020/1cb609e0733fbb9822d203ff8bcc0003 to your computer and use it in GitHub Desktop.
Web Scraper sitemap for Square Enix Store. Here I have made the scroller separate from the data scraper (usually the data scraper is a child of the scroller).
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| { | |
| "_id": "forum-square-enix-store", | |
| "startUrl": ["https://store.na.square-enix-games.com/en_US/merchandise/all-merchandise"], | |
| "selectors": [{ | |
| "id": "Separate scroller", | |
| "type": "SelectorElementScroll", | |
| "parentSelectors": ["_root"], | |
| "selector": "a.product-link-box", | |
| "multiple": true, | |
| "delay": "2100" | |
| }, { | |
| "id": "Product links", | |
| "type": "SelectorLink", | |
| "parentSelectors": ["_root"], | |
| "selector": "a.product-link-box", | |
| "multiple": true, | |
| "delay": 0 | |
| }] | |
| } |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
TIP: before you start scraping, load the Url you want, then zoom out (reduce) the browser view to 50%. You can do this by pressing Ctrl - (Ctrl-minus) or by using the Chrome settings (click on the "3 dots" -> Zoom). This will force the browser to load more tiles and will also help the scroller to work better. Remember, you will need to perform this zoom out step before you start the scraper.