Skip to content

Instantly share code, notes, and snippets.

@szeidner
Last active September 5, 2017 12:51
Show Gist options
  • Save szeidner/9967e76b19eff2157602afb8663273b9 to your computer and use it in GitHub Desktop.
Save szeidner/9967e76b19eff2157602afb8663273b9 to your computer and use it in GitHub Desktop.
Fetch feed data
/// fetch_all_feed_data
/// async fetch to get all feeds from urls
pub fn fetch_all_feed_data(&mut self) -> Result<Vec<Channel>, Error> {
let requests: Vec<_> = self.get_all_feed_urls()
.into_iter()
.take(50)
.filter_map(|url| match url.parse::<hyper::Uri>() {
Ok(uri) => {
println!("{:?}", url);
Some(uri)
},
Err(error) => {
println!("Error parsing URL: {:?}, {:?}", error, url);
None
}
})
.map(|url|
self.client.get(url)
.and_then(|response| response.body().concat2())
.and_then(move |body| {
let channel = Channel::read_from(body.as_ref()).unwrap_or(Channel::default());
Ok(channel)
}))
.collect();
self.core.run(join_all(requests))
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment