Skip to content

Instantly share code, notes, and snippets.

@Narsil
Created May 22, 2020 18:11
Show Gist options
  • Save Narsil/833ef43fe4e6169cf54de57733a753b5 to your computer and use it in GitHub Desktop.
Save Narsil/833ef43fe4e6169cf54de57733a753b5 to your computer and use it in GitHub Desktop.
Small example to get lower bound of word counting of a file.
use rayon::prelude::*;
use std::collections::HashMap;
fn main() {
let buf = std::fs::read_to_string("data/big.txt").unwrap();
let word_count = buf
.par_lines()
.map(|line| {
let mut frequency: HashMap<&str, u32> = HashMap::new();
for word in line.split(' ') {
*frequency.entry(word).or_insert(0) += 1;
}
frequency
})
.reduce(HashMap::new, |mut acc, freq| {
for (k, v) in freq {
*acc.entry(k).or_insert(0) += v;
}
acc
});
println!("Word count {}", word_count.len());
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment