Skip to content

Instantly share code, notes, and snippets.

@slawosz

slawosz/wiki.rb

Last active August 29, 2015 14:15
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save slawosz/1ccf272330f119a8a02a to your computer and use it in GitHub Desktop.
Save slawosz/1ccf272330f119a8a02a to your computer and use it in GitHub Desktop.
Download ZIP
Raw
wiki.rb
require 'open-uri'
require 'lingua/stemmer'
stemmer= Lingua::Stemmer.new(:language => "de")
words = doc.css("#mw-content-text").text.gsub("\n","").gsub("|","").gsub("\d{1,}", "").split(" ").select { |w| w.length > 2 }.map { |w| w.chomp }
h = Hash.new { |k,v| k[v] = 0 }
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment