Skip to content

Instantly share code, notes, and snippets.

View gist:293baec4ff71da0f10e1d212aed078a0
<title>([^<]*)
document.body.innerHTML = document.body.innerText
@sysnucleus
sysnucleus / webharvy-takealot
Created Feb 6, 2022
WebHarvy takealot.com codes
View webharvy-takealot
// JavaScript Code
els = document.getElementsByClassName('image-box undefined');
for (var i = els.length - 1; i >= 0; i--) {
img = els[i].children[0];
img.setAttribute('src', img.getAttribute('src').replace('list', 'pdpxl'));
}
// RegEx
View WebHarvy XML Miner Options
<MinerOptions>
<MinLevelsUp>2</MinLevelsUp>
<MinChildCount>10</MinChildCount>
<SelAccuracy>2</SelAccuracy>
</MinerOptions>
@sysnucleus
sysnucleus / WebHarvy XML Version Info
Last active Sep 9, 2021
WebHarvy XML Version Info
View WebHarvy XML Version Info
<VersionInfo>6.3.0.189</VersionInfo>
<RegInfo>SysNucleus</RegInfo>
@sysnucleus
sysnucleus / ta-expand.js
Created Apr 5, 2021
Expand Tripadvisor reviews 'Read More' link..
View ta-expand.js
els = document.getElementsByTagName('span');
for (var i = els.length - 1; i >= 0; i--) {
if(els[i].innerText === 'Read more') {
els[i].click();
}
}
@sysnucleus
sysnucleus / tripadvisor
Created Sep 3, 2020
Codes to extract reviewer submitted images from TripAdvisor using WebHarvy
View tripadvisor
// RegEx to Follow links
href="([^"]*)
// More button click
document.getElementsByClassName('moreBtn')[0].click();
// Get images block
@sysnucleus
sysnucleus / gist:436a2b0be80882f0ae61a391931abf5d
Created Aug 31, 2020
RegEx strings to extract email, phone, website and address from yellowpages.com.au
View gist:436a2b0be80882f0ae61a391931abf5d
data-email="([^"]*)
tel:([^"]*)
title="([^\s]*)\s*\(opens in a new window\)
<p class="listing-address[^>]*>([^<]*)
@sysnucleus
sysnucleus / gist:84a0574cbf908813787d2d95b8a6c2ed
Created Aug 20, 2020
JS code to configure pagination (scroll) in WebHarvy for Twitter scraping
View gist:84a0574cbf908813787d2d95b8a6c2ed
groupEl = document.getElementsByTagName('article')[0].parentElement.parentElement.parentElement.parentElement;
groupEl.children[groupEl.childElementCount-1].scrollIntoView();
@sysnucleus
sysnucleus / ebay
Created Jun 16, 2019
RegEx to extract email/phone from eBay sellers page
View ebay
id="email"[\s\S]*?cell_value">([^<]*)
id="phone_number"[\s\S]*?cell_value">([^<]*)
@sysnucleus
sysnucleus / yellow pages egypt.js
Created Oct 15, 2018
RegEx strings to extract listing name, telephone, website and address
View yellow pages egypt.js
[\s]*(.*)
tel: ([^"]*)
title="Website"[\s\S]*?href="([^"]*)
class="col-md-9 company_address"[^>]*>([^<]*)