Skip to content

Instantly share code, notes, and snippets.

@LeeMeng2020
Created September 10, 2020 13:41
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save LeeMeng2020/5492e55fcf6f02edde8d8e9de12edd31 to your computer and use it in GitHub Desktop.
Save LeeMeng2020/5492e55fcf6f02edde8d8e9de12edd31 to your computer and use it in GitHub Desktop.
This'll click all the Show Email buttons and scrape the emails. There's a 750 ms (0.75 sec) delay between each click so it'll take about 15 sec to complete for the example search. I used Page load delay (ms): 6000.
{
"_id": "yellowpages-co-za",
"startUrl": ["https://www.yellowpages.co.za/search?what=accounting+services&where=pinetown"],
"selectors": [{
"id": "listing wrappers",
"type": "SelectorElement",
"parentSelectors": ["_root"],
"selector": "div.yp-object-result-item",
"multiple": true,
"delay": 0
}, {
"id": "Click Show Email",
"type": "SelectorElementClick",
"parentSelectors": ["listing wrappers"],
"selector": "div.pb-1",
"multiple": false,
"delay": "750",
"clickElementSelector": "b.idShowEmail",
"clickType": "clickOnce",
"discardInitialElements": "do-not-discard",
"clickElementUniquenessType": "uniqueText"
}, {
"id": "Company name",
"type": "SelectorText",
"parentSelectors": ["listing wrappers"],
"selector": "h5",
"multiple": false,
"regex": "",
"delay": 0
}, {
"id": "Email",
"type": "SelectorText",
"parentSelectors": ["Click Show Email"],
"selector": "div .fullDetailId a[onclick*=Email]",
"multiple": false,
"regex": "",
"delay": 0
}]
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment