Skip to content

Instantly share code, notes, and snippets.

@monajalal
Created August 23, 2013 23:39
Show Gist options
  • Save monajalal/6325015 to your computer and use it in GitHub Desktop.
Save monajalal/6325015 to your computer and use it in GitHub Desktop.
def token_stemming(tokens, suffixes):
suffixes.sort(key=len,reverse=True)
for token in tokens:
counter=0
for suffix in suffixes:
if token[(len(token)-len(suffix)): ]==suffix:
print token[:(len(token)-len(suffix))]
break
else :
counter+=1
if counter==len(suffixes):
#print counter
print token
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment