Skip to content

Instantly share code, notes, and snippets.

@mrm1001
Created August 11, 2018 16:51
Show Gist options
  • Save mrm1001/280c4477088d5b4447b9f638f518f08c to your computer and use it in GitHub Desktop.
Save mrm1001/280c4477088d5b4447b9f638f518f08c to your computer and use it in GitHub Desktop.
FastText
void Dictionary::addWordNgrams(std::vector<int32_t>& line,
const std::vector<int32_t>& hashes,
int32_t n) const {
for (int32_t i = 0; i < hashes.size(); i++) {
uint64_t h = hashes[i];
for (int32_t j = i + 1; j < hashes.size() && j < i + n; j++) {
h = h * 116049371 + hashes[j];
pushHash(line, h % args_->bucket);
}
}
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment