Skip to content

Instantly share code, notes, and snippets.

@kxzk
Last active January 5, 2018 22:24
Show Gist options
  • Save kxzk/191f7278861ce549430e21dfbcea0cd2 to your computer and use it in GitHub Desktop.
Save kxzk/191f7278861ce549430e21dfbcea0cd2 to your computer and use it in GitHub Desktop.
Grab all headlines from Hacker News into a vector
extern crate reqwest;
extern crate scraper;
// importation syntax
use scraper::{Html, Selector};
fn main() {
hn_headlines("https://news.ycombinator.com");
}
fn hn_headlines(url: &str) {
let mut resp = reqwest::get(url).unwrap();
assert!(resp.status().is_success());
let body = resp.text().unwrap();
// parses string of HTML as a document
let fragment = Html::parse_document(&body);
// parses based on a CSS selector
let stories = Selector::parse(".storylink").unwrap();
// iterate over elements matching our selector
for story in fragment.select(&stories) {
// grab the headline text and place into a vector
let story_txt = story.text().collect::<Vec<_>>();
println!("{:?}", story_txt);
}
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment