Skip to content

Instantly share code, notes, and snippets.

@hkasera
Last active September 9, 2017 20:43
Show Gist options
  • Save hkasera/663bcfc9ec84877426a9cb9545d99f65 to your computer and use it in GitHub Desktop.
Save hkasera/663bcfc9ec84877426a9cb9545d99f65 to your computer and use it in GitHub Desktop.
A node js script to get the top 5 results of given queries in Bing and Google

Installation

$ Install Node.js (Puppeteer requires at least Node v6.4.0)

$ Install puppeteer

$ node query.js > output.txt

Query Term : Reighan Gillam Dornsife
Bing Results :
https://dornsife.usc.edu/cf/faculty-and-staff/faculty.cfm?pid=1080560
https://news.usc.edu/119860/milton-curry-named-new-dean-at-usc-school-of-architecture/
http://dailytrojan.com/2017/04/10/milton-curry-to-serve-as-usc-dean-of-architecture/
http://dornsife.usc.edu/cf/anth/faculty_display.cfm?person_id=1080560
https://lasentinel.net/milton-curry-named-new-dean-at-usc-school-of-architecture.html
Google Results :
https://lsa.umich.edu/daas/people/core-faculty/reighan.html
http://dornsife.usc.edu/cf/anth/faculty_display.cfm?person_id=1080560
https://dornsife.usc.edu/cf/faculty-and-staff/faculty-by-dept.cfm?did=6
http://reighangillam.weebly.com/
http://dailytrojan.com/2017/04/10/milton-curry-to-serve-as-usc-dean-of-architecture/
Query Term : Janel Hoskins Dornsife
Bing Results :
https://dornsife.usc.edu/vsri/Janet-Hoskins
https://dornsife.usc.edu/dr-janet-hoskins/
https://dornsifecms.usc.edu/dr-janet-hoskins/books/
http://mugshots.com/US-Counties/Kentucky/Clay-County-KY/Janel-Hoskins.134962136.html
https://www.facebook.com/public/Janel-Hoskins
Google Results :
http://catalogue2012.usc.edu/files/2012/07/college.pdf
http://dornsife.usc.edu/news/stories/272/gaining-a-political-voice-via-religion/
https://dornsife.usc.edu/ase/news/
https://drexel.edu/~/media/Files/dornsife/Faculty%20CVs/Martinez-Donate%20CV.ashx?la=en
https://hacc.edu/Students/FacultyandStaffDirectory/Faculty-and-Staff-Directory_act.cfm?lpha=D
Query Term : Lanita Jacobs Dornsife
Bing Results :
https://dornsife.usc.edu/lanita-jacobs/
https://pressroom.usc.edu/lanita-jacobs/
https://dornsifecms.usc.edu/news/topics/8817/lanita-jacobs/
http://www-ccd.usc.edu/news/topics/8817/lanita-jacobs/
https://dornsifecms.usc.edu/anth/faculty-research-areas/
Google Results :
https://dornsife.usc.edu/lanita-jacobs/research/
https://dornsife.usc.edu/cf/faculty-and-staff/faculty.cfm?pid=1003379
https://dornsife.usc.edu/lanita-jacobs/classes
https://dornsife.usc.edu/lanita-jacobs
https://pressroom.usc.edu/lanita-jacobs/
Query Term : Art History Dornsife
Bing Results :
http://dornsife.usc.edu/ahis/
https://gradadm.usc.edu/our-programs/dornsife-college-of-letters-arts-and-sciences/
https://dornsifecms.usc.edu/news/topics/151/art-history/
http://dornsifecms.usc.edu/megan-mastroianni/
http://www.findglocal.com/US/Los-Angeles/545647998822481/USC-Dornsife-Department-of-Art-History
Google Results :
https://dornsife.usc.edu/ahis/
http://dornsife.usc.edu/ahis/faculty-roster/
https://dornsife.usc.edu/ahis/contact-us/
https://dornsife.usc.edu/ahis/graduate-studies-new/
http://dornsife.usc.edu/Art-History-overseas/
Query Term : Classics Dornsife
Bing Results :
https://dornsife.usc.edu/clas
https://dornsife.usc.edu/clas
https://dornsife.usc.edu/classics-overseas/
https://dornsifecms.usc.edu/news/topics/844/classics/
https://dornsifecms.usc.edu/aileen-ng/
Google Results :
https://dornsife.usc.edu/clas/
http://dornsife.usc.edu/clas/faculty/
https://dornsife.usc.edu/clas/undergraduate/
https://dornsife.usc.edu/clas/degree-requirements/
https://dornsife.usc.edu/clas/contact-us/
Query Term : Comparative Literature Dornsife
Bing Results :
https://dornsife.usc.edu/colt/
https://dornsife.usc.edu/comparative-literature-overseas/
https://dornsifecms.usc.edu/colt/courses/
https://dornsifecms.usc.edu/colt/honors/
https://dornsifecms.usc.edu/colt/events/
Google Results :
https://dornsife.usc.edu/colt/
http://dornsife.usc.edu/colt/people/
https://dornsife.usc.edu/colt/undergraduate/
https://dornsife.usc.edu/colt/graduate/
https://dornsife.usc.edu/colt/courses/
Query Term : Dornsife USC map
Bing Results :
http://dornsife.usc.edu/map/
http://dornsife.usc.edu/sitemap/
http://spatial.usc.edu/index.php/research-innovation/maps-maps-processing/
https://dornsifecms.usc.edu/news/stories/1866/map-your-future/
https://gradadm.usc.edu/our-programs/dornsife-college-of-letters-arts-and-sciences/
Google Results :
https://dornsife.usc.edu/assets/sites/1/docs/about/CBO_Map.pdf
http://spatial.usc.edu/index.php/research-innovation/maps-maps-processing/
https://web-app.usc.edu/maps/map.pdf
http://www.usc.edu/maps
https://dornsife.usc.edu/csii/eligible-to-naturalize-map/
Query Term : David and Dana Dornsife
Bing Results :
https://dornsife.usc.edu/dornsife-about/
http://dornsife.usc.edu/dornsife
https://www.philanthropy.com/article/No-7-DavidDana-Dornsife/157247
https://en.wikipedia.org/wiki/David_H._Dornsife
https://dornsifecms.usc.edu/news/topics/30145/david-and-dana-dornsife/
Google Results :
https://dornsife.usc.edu/dornsife-about/
https://dornsife.usc.edu/dornsife
https://en.wikipedia.org/wiki/David_H._Dornsife
http://articles.latimes.com/2011/mar/09/local/la-me-0309-usc-gift-20110309
https://www.philanthropy.com/article/No-7-DavidDana-Dornsife/157247
Query Term : USC Dornsife Alumni
Bing Results :
http://dornsife.usc.edu/alumni/
https://usccareers.usc.edu/job/los-angeles/director-of-dornsife-alumni-relations/1209/5193501
https://dornsifecms.usc.edu/linguistics-alumni/
http://spatial.usc.edu/index.php/alumni/
http://dornsife.usc.edu/to-alumni/
Google Results :
https://dornsife.usc.edu/alumni/
http://spatial.usc.edu/index.php/alumni/
http://dornsife.usc.edu/meet-alumni/
https://dornsife.usc.edu/featured-alum/
https://alumni.usc.edu/
Query Term : USC Mathematics Undergraduate degree requirements
Bing Results :
https://dornsife.usc.edu/mathematics/degree-requirements/
http://dornsife.usc.edu/mathematics/graduate-degree-requirements/
http://catalogue.usc.edu/content.php?catoid=2&navoid=400
http://ee.usc.edu/info/studentresources/EE_Catalogue.html
http://catalogue.usc.edu/content.php?catoid=2&navoid=263
Google Results :
https://dornsife.usc.edu/mathematics/degree-requirements/
https://dornsife.usc.edu/mathematics/General-info/
http://catalogue.usc.edu/content.php?catoid=2&navoid=400
https://dornsife.usc.edu/mathematics/undergraduate-course-descriptions/
https://dornsife.usc.edu/mathematics/learning/
Query Term : USC Mathematics Masters degree requirements
Bing Results :
https://dornsife.usc.edu/mathematics/graduate-degree-requirements/
https://dornsife.usc.edu/mathematics/degree-requirements/
http://catalogue.usc.edu/content.php?catoid=2&navoid=263
https://dornsife.usc.edu/masters-degrees/
http://catalogue.usc.edu/content.php?catoid=2&navoid=400
Google Results :
https://dornsife.usc.edu/mathematics/graduate-general-information/
https://dornsife.usc.edu/mathematics/graduate-degree-requirements/
https://dornsife.usc.edu/mathematics/admissions
https://dornsife.usc.edu/mathematics/degree-requirements/
https://dornsife.usc.edu/mathematics/applied-mathematics
Query Term : USC Mathematics Ph.D. degree requirements
Bing Results :
https://dornsife.usc.edu/mathematics/graduate-degree-requirements/
https://dornsife.usc.edu/mathematics/degree-requirements/
http://catalogue.usc.edu/content.php?catoid=2&navoid=400
https://cinema.usc.edu/admissions/procedures/mediastudies/phdprocedures.cfm
http://catalogue.usc.edu/preview_program.php?catoid=6&poid=5366&returnto=1424
Google Results :
https://dornsife.usc.edu/mathematics/graduate-general-information/
http://catalogue.usc.edu/preview_program.php?catoid=2&poid=1255
https://dornsife.usc.edu/mathematics/graduate-degree-requirements/
https://dornsife.usc.edu/mathematics/admissions
https://dornsife.usc.edu/mathematics/applied-mathematics
Query Term : USC village retail shops list
Bing Results :
https://usc-village.com/about/
http://villagearchive.usc.edu/
https://news.usc.edu/tag/usc-village/
https://villagearchive.usc.edu/faq/retail/
https://usc-village.com/tag/shops-at-usc-village/
Google Results :
https://news.usc.edu/126519/whats-in-store-introducing-the-restaurants-services-and-stores-at-usc-village/
https://realestate.usc.edu/uscvillage/shops/
https://thetab.com/us/usc/2017/07/18/the-10-new-storefronts-opening-in-the-usc-village-5600
https://thetab.com/us/usc/2017/07/17/the-15-new-restaurants-opening-in-the-usc-village-5589
http://dailytrojan.com/2017/05/23/usc-village-adds-vendors-nears-completion/
const puppeteer = require('puppeteer');
var queries = ["Reighan Gillam Dornsife",
"Janel Hoskins Dornsife",
"Lanita Jacobs Dornsife",
"Art History Dornsife",
"Classics Dornsife",
"Comparative Literature Dornsife",
"Dornsife USC map",
"David and Dana Dornsife",
"USC Dornsife Alumni",
"USC Mathematics Undergraduate degree requirements",
"USC Mathematics Masters degree requirements",
"USC Mathematics Ph.D. degree requirements",
"USC village retail shops list"]
var bingSearchURL = "http://www.bing.com/search?q=";
var googleSearchURL = "http://www.google.com/search?q=";
(async () => {
const browser = await puppeteer.launch();
const page = await browser.newPage();
page.on('console', (...args) => console.log(...args));
for(let query of queries) {
console.log("Query Term : "+ query+ "\n");
var queryTerm = query.split(" ").join("+");
var bingSearchQuery = bingSearchURL + queryTerm;
var googleSearchQuery = googleSearchURL + queryTerm;
await page.goto(bingSearchQuery);
console.log("Bing Results : \n");
await page.evaluate(() => {
var c = document.getElementsByClassName("b_algo");
var output = "";
for(var i = 0; i < 5 ; ++ i ) {
var output = output + c[i].childNodes[0].childNodes[0].getAttribute("href") ;
output = output + "\n";
}
console.log(output);
});
await page.goto(googleSearchQuery);
console.log("Google Results : \n");
await page.evaluate(() => {
var c = document.getElementsByClassName("r");
var output = ""
for(var i = 0; i < 5 ; ++ i ) {
var aTag = c[i].getElementsByTagName('a')[0];
var datahref = aTag.getAttribute("data-href");
var href = aTag.getAttribute("href");
if(datahref == null) {
output = output + href;
} else {
output = output + datahref;
}
output = output + "\n";
}
console.log(output);
});
};
browser.close();
})();
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment