Skip to content

Instantly share code, notes, and snippets.

@geroldcsendes
geroldcsendes / res_df.csv
Created November 15, 2019 16:39
AWS Comprehend result table
Index Sentiment Mixed Negative Neutral Positive
1 POSITIVE 9.932292414305266e-6 0.010471675544977188 0.2103179544210434 0.7792003750801086
@geroldcsendes
geroldcsendes / detect_sentiment.R
Last active November 15, 2019 16:46
Function to detect sentiment from a dataframe
sentiment <- function(row, df) {
record <- df[row,]
sentiment <- detect_sentiment(as.character(record$plain_tweet)) # Get sentiment from Amazon's Comprehemd
merged <- merge(sentiment, record) # Merge the sentiment result to the original data
return (merged)}
row_seq <- seq(1,nrow(trainer)) # Define argument for lapply
sentiment <- lapply(row_seq, sentiment, df=trainer)
appended_df4 <- rbindlist(sentiment) # Append list of dataframes together
@geroldcsendes
geroldcsendes / estimate_cost.R
Created November 15, 2019 14:07
Estimate Amazon Comprehend cost
estimate_cost <- function(df) {
# glue library is needed for this function
df <- transform(df, chars = ifelse(nchar(plain_tweet) > 300, nchar(plain_tweet), 300))
cost_estimate <- round(((sum(try_calc$chars))/(100)*0.0001),2)
return_string <- glue("The estimated price for this sentiment analysis is {cost_estimate} dollars")
print(return_string)
}
@geroldcsendes
geroldcsendes / twitter_sentiment.R
Last active November 15, 2019 15:45
Get twitter data, clean & analyze it
# Get twitter data
trainer <- search_tweets(q = "Guardiola", n=18000, type='mixed', lang="en")
# Source for URL removal: https://www.earthdatascience.org/courses/earth-analytics/get-data-using-apis/text-mining-twitter-data-intro-r/
trainer$stripped_text <- gsub("http.*","", trainer$text) # Remove http
trainer$stripped_text <- gsub("https.*","", trainer$stripped_text) # Remove https
# Emoji removal
trainer$plain_tweet <- enc2native(trainer$stripped_text) # Covnert emojis to native encoding
trainer$plain_tweet <- gsub("<.*.>", "", trainer$plain_tweet)
trainer$plain_tweet <- trimws(trainer$plain_tweet) # Remove leading whitespaces from the beginning
@geroldcsendes
geroldcsendes / AWS_setup.R
Last active November 15, 2019 13:22
Set up AWS in R
# Install package if needed: install.packages("aws.comprehend")
# Read in access key generated in IAM Console
keyTable <- read.csv("accessKeys.csv", header = T) # accessKeys.csv == the CSV downloaded from AWS containing your Acces & Secret keys
# Assign keys to env variable
AWS_ACCESS_KEY_ID <- as.character(keyTable$Access.key.ID)
AWS_SECRET_ACCESS_KEY <- as.character(keyTable$Secret.access.key)
#A ctivate keys
Sys.setenv("AWS_ACCESS_KEY_ID" = AWS_ACCESS_KEY_ID,
"AWS_SECRET_ACCESS_KEY" = AWS_SECRET_ACCESS_KEY,
"AWS_DEFAULT_REGION" = "eu-west-1")
@geroldcsendes
geroldcsendes / create_token.R
Last active November 15, 2019 13:20
Setting up rtweet
appname <- "ceu_football_managers"
key <- <YOUR_KEY_HERE>
secret_key <- <YOUR_SECRET_KEY_HERE>
access_token <- <YOUR_ACCESS_TOKEN_HERE>
access_secret <- <YOUR_SECRET_ACCESS_TOKEN_HERE>
# Create token named "twitter_token"
twitter_token <- create_token(
app = appname,
consumer_key = key,
consumer_secret = secret_key,