Skip to content

Instantly share code, notes, and snippets.

@aquasmit
Created April 6, 2016 16:28
Show Gist options
  • Save aquasmit/7f4e93f7a7f1cb13c487b429dd603dde to your computer and use it in GitHub Desktop.
Save aquasmit/7f4e93f7a7f1cb13c487b429dd603dde to your computer and use it in GitHub Desktop.
TripAdvisor Reviews Scraping using X-Ray (Node.js)
var Xray = require('x-ray');
var x = Xray();
x('https://www.tripadvisor.com/Hotel_Review-g60763-d93545-Reviews-The_Manhattan_at_Times_Square_Hotel-New_York_City_New_York.html', '#REVIEWS .reviewSelector', [{
reviewer: '.username.mo span',
//ajax_link:'div#most_recent_pager div .pam.uiBoxWhite.noborder.uiMorePagerPrimary@href'
reviewer_location: '.location',
//review_rating: '.rating-very-large meta[content]@content',
//review_date: '.rating-qualifier [content]@content',
review_content:'.reviewSelector .innerBubble .wrap .reviewItemInline + .entry p.partial_entry'
}])
.paginate('.nav.next.rndBtn.taLnk@href')
//.limit(300)
.write('results.json')
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment