Skip to content

Instantly share code, notes, and snippets.

@LeeMeng2020
Last active October 3, 2020 03:46
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save LeeMeng2020/f7c3f20210783b2e0219edc6deddd000 to your computer and use it in GitHub Desktop.
Save LeeMeng2020/f7c3f20210783b2e0219edc6deddd000 to your computer and use it in GitHub Desktop.
This site uses random attribute names like sc-hkbPbT so better selectors are needed. The sitemap below which will expand all the accordions and get the links. To make it click on all links you'll need to add data scrapers under "Get links" (currently it will just get the URLs and not click thru): Originally posted at: https://forum.webscraper.io…
{
"_id": "adac-test",
"startUrl": ["https://www.adac.de/rund-ums-fahrzeug/autokatalog/marken-modelle/citroen/berlingo/2generation-facelift-2/"],
"selectors": [{
"id": "Open accordians",
"type": "SelectorElementClick",
"parentSelectors": ["_root"],
"selector": "main div[class^='sc']:contains('Fahrzeuge') div[role='button'] ~ div[class^='sc']",
"multiple": true,
"delay": "2100",
"clickElementSelector": "main div[class^='sc']:contains('Modelle') div[class^='sc'] > div[class^='sc']:contains('Fahrzeuge')",
"clickType": "clickOnce",
"discardInitialElements": "do-not-discard",
"clickElementUniquenessType": "uniqueText"
}, {
"id": "Get links",
"type": "SelectorLink",
"parentSelectors": ["Open accordians"],
"selector": "a[data-tracking]",
"multiple": true,
"delay": 0
}]
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment