Skip to content

Instantly share code, notes, and snippets.

@grantglidewell
Last active December 5, 2018 13:14
Show Gist options
  • Save grantglidewell/ae55169b990673cbd6c33cb37b5d5f51 to your computer and use it in GitHub Desktop.
Save grantglidewell/ae55169b990673cbd6c33cb37b5d5f51 to your computer and use it in GitHub Desktop.
extern crate reqwest;
use std::time::SystemTime;
use std::fs::File;
use std::io::prelude::*;
fn write_file(filename: &str, data: &str) {
let bytes = data.as_bytes();
let mut file = File::create(filename).expect("Error creating file");
file.write_all(bytes).expect("Error writing file");
}
fn read_file(filename: &str) -> String {
let mut file = File::open(filename).expect("Error opening file");
let mut contents = String::new();
file.read_to_string(&mut contents);
return contents;
}
fn gen_timestamp() -> String {
match SystemTime::now().duration_since(SystemTime::UNIX_EPOCH) {
Ok(n) => n.as_secs().to_string(),
Err(_) => panic!("SystemTime before UNIX EPOCH!"),
}
}
fn scrape(url: &str, term: &str) -> String {
// Make the http request for the resource
// and assign the text response to a variable
let response_text = reqwest::get(url)
.expect("Unable to find url")
.text()
.expect("Unable to read response");
// Create a unique token for filename
let timestamp = gen_timestamp();
let filename = ["scape", &timestamp, ".html"].join("");
// Write the response text to disk
write_file(&filename, &response_text);
// Read the file to a variable
let scrape_data = read_file(&filename);
// Check if the search term is present
if scrape_data.contains(term) {
// Return a message to the user that the search term was found
return ["Found", term, "in", url].join(" ");
}
// Return a message to the user that the search term was not found
return [term, "was not found in", url].join(" ");
}
fn main() {
println!("{}", scrape("URL", "SEARCH_TERM"));
}
@brodeuralexis
Copy link

Some small idiomatic improvements:
Line 20: The match arms are doing the same thing as expect.
Line 37, 49 and 53: Use format! from std::format.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment