Skip to content

Instantly share code, notes, and snippets.

View Arf9999's full-sized avatar

Andrew Fraser Arf9999

View GitHub Profile
@Arf9999
Arf9999 / cleanup_crime_stats.R
Last active November 7, 2019 12:38
Cleanup of South African 2019 crime statistics into a two tidy formats
library (tidyverse)
library(readxl)
library(httr)
library(janitor)
##Download crime stats from SAPS site
url1 <-
"https://www.saps.gov.za/services/2018-2019_crimestatistics.xlsx" #URL of the SAPS excel file
GET(url1, write_disk(temp <-
tempfile(fileext = ".xlsx"))) # save the xlsx file to disk temporarily
##--------------------------------------------------------------------------------------------------##
##Mapping neo-nazi forum post volume to world map using the Iron March Forum leaked SQL database ##
##Leaked database is available here: https://archive.org/details/iron_march_201911 ##
## All below packages are required. All are available on CRAN. ##
##--------------------------------------------------------------------------------------------------##
##packages--------##
##function to rehydrate GetOldTweets3 tweet data using rtweet & standard API
##NB: A twitter auth token or alternatively a list of tokens that are loaded into the environment *must* be specified
rehydrate_got3_statuses <- function(got3_df,token = NULL, token_list = NULL ) {
require(rtweet, quietly = TRUE)
require(dplyr, quietly = TRUE)
require(purrr, quietly = TRUE)
require(readr, quietly = TRUE)
library (tidyverse)
library(readxl)
library(httr)
library(janitor)
##Download crime stats from SAPS site
url1 <-
"https://www.saps.gov.za/services/Crime-Statistics%202019_2020.xlsx" #URL of the SAPS excel file
GET(url1, write_disk(temp <-
tempfile(fileext = ".xlsx"))) # save the xlsx file to disk temporarily
@Arf9999
Arf9999 / got_wrapper.R
Last active August 31, 2020 10:01
Wrapper for GetOldTweets command line query.
## Run once per session function to make sure Python is available from R, install GOT3 into environment, make sure {reticulate} library is loaded.
prepare_got <- function(){
require(reticulate)
library(reticulate)
reticulate::conda_install(packages = "GetOldTweets3", pip = TRUE)
}
@Arf9999
Arf9999 / snscrape_twitter_search.R
Created October 14, 2020 09:06
twitter search wrapper for old tweets using snscrape
#################################################################
##Requires python 3.8 and snscrape dev version to be installed ##
##see https://github.com/JustAnotherArchivist/snscrape ##
#################################################################
snscrape_search <- function (search_string, #search terms in quotes
##download pdfs of Eskom Air Quality reports from web.
require(tidyverse) #we use purrr, stringr and dplyr
require(remotes)
URLS <- tibble::tribble(
~`url`,
"https://www.eskom.co.za/Whatweredoing/AirQuality/Pages/ArnotEmissions.aspx",
"https://www.eskom.co.za/Whatweredoing/AirQuality/Pages/Camden-Power-Station.aspx",
"https://www.eskom.co.za/Whatweredoing/AirQuality/Pages/Duvha-Power-Station.aspx",
#'@title Twitter matrix of friend connection
#'Code modded from boB Rudis' 21 Recipes for Mining Twitter with Rtweet
#'https://rud.is/books/21-recipes/visualizing-intersecting-follower-sets-with-upsetr.html
#'@description This function creates a matrix of friends of a list of twitter users,
#' sums the number of common friends, and then ranks them in descending order.
#'@param user_list A list of user_names
#'@param friend_depth The number of most recent friends to include. Defaults to 200
#'@param token A twitter oauth token. Default is NULL and will use token in Environment
#'@keywords twitter, rtweet, visualization, friend analysis
#'@export
@Arf9999
Arf9999 / aoc2021_1.R
Last active December 1, 2021 12:18
Advent of Code 2021 - day1
##Advent of code Day1
require(tidyverse)
input <- tibble::tribble(
~depth,
132L,
146L,
153L,
175L,