Skip to content

Instantly share code, notes, and snippets.

@aaronkub
Last active April 11, 2019 14:44
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save aaronkub/c9b39a96c50622a1e61aa9ff343e83b8 to your computer and use it in GitHub Desktop.
Save aaronkub/c9b39a96c50622a1e61aa9ff343e83b8 to your computer and use it in GitHub Desktop.
def get_stemmed_text(corpus):
from nltk.stem.porter import PorterStemmer
stemmer = PorterStemmer()
return [' '.join([stemmer.stem(word) for word in review.split()]) for review in corpus]
stemmed_reviews = get_stemmed_text(reviews_train_clean)
@deadania14
Copy link

hy aaron. i want to ask about this code. the return is list right? what the review.split() for? tokenize? i confuse because this the result i got :

review = "When it works it's great but for some reason it doesn't always answer me When it doesn't answer me I turn off the tv and make sure no one else is talking and it still doesn't work at that time. It was easy to set up with the phone app. Doesn't seem to matter where the phone is when it isn't working. I've tried close and far and in between. So a case of great when it works and the pits when it doesn't"
def get_stemmed_text(corpus):
return [' '.join([stemmer.stem(word) for word in review.split()]) for review in corpus]

stemmed_reviews = get_stemmed_text(review)

['W',
'h',
'e',
'n',
'',
'i',
't',
'',
'w',
'o',
'r',
'k',
's',
'',
'i',
't',
"'",
's',
'',
'g',
'r',
'e',
'a',
't',
'',
'b',
'u',
't',
'',
'f',
'o',
'r',
'',
's',
'o',
'm',
'e',
'',
'r',
'e',
'a',
's',
'o',
'n',
'',
'i',
't',
'',
'd',
'o',
'e',
's',
'n',
"'",
't',
'',
'a',
'l',
'w',
'a',
'y',
's',
'',
'a',
'n',
's',
'w',
'e',
'r',
'',
'm',
'e',
'',
'W',
'h',
'e',
'n',
'',
'i',
't',
'',
'd',
'o',
'e',
's',
'n',
"'",
't',
'',
'a',
'n',
's',
'w',
'e',
'r',
'',
'm',
'e',
'',
'I',
'',
't',
'u',
'r',
'n',
'',
'o',
'f',
'f',
'',
't',
'h',
'e',
'',
't',
'v',
'',
'a',
.....

could you explain why?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment