dbrby
/ fifth_senedd_df.R
Last active
August 2, 2021 16:38
Build dataset from legislative speech records from the Fifth Senedd
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| require(rvest) | |
| require(lubridate) | |
| require(tidyverse) | |
| require(readr) | |
| require(data.table) | |
| require(textclean) | |
| require(XML) | |
| require(lubridate) | |
| require(pbapply) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| require(rvest) | |
| require(tidyverse) | |
| require(readr) | |
| require(data.table) | |
| require(XML) | |
| require(lubridate) | |
| require(pbapply) | |
| # https://record.senedd.wales/XMLExport/?start=2016-05-05&end=2017-06-05 (results page) |
dbrby
/ fifth_senedd_members.R
Created
August 5, 2021 14:03
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| require(rvest) | |
| require(pbapply) | |
| require(tidyverse) | |
| require(tidywikidatar) | |
| base_url <- "https://en.wikipedia.org/" | |
| pg <- read_html("https://en.wikipedia.org/wiki/Members_of_the_5th_National_Assembly_for_Wales") | |
dbrby
/ wordfish_afghanistan_2021_08_19.R
Created
August 19, 2021 02:42
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| require(rvest) | |
| require(purrr) | |
| require(textclean) | |
| require(tidyverse) | |
| require(hansard) | |
| require(quanteda) | |
| require(quanteda.textmodels) | |
| require(quanteda.textplots) | |
| pg <- read_html("https://hansard.parliament.uk/commons/2021-08-18/debates/A86142BD-A204-4BC8-BBC0-ACA7BAD7E9F0/Afghanistan") |
dbrby
/ hansard_grabber.R
Last active
August 21, 2021 14:27
Scrape all legislative speeches from Hansard
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| require(purrr) | |
| require(tidyverse) | |
| elect_2010 <- as.Date("2010-05-08") # Storing for parameters on future API calls | |
| x <- seq.Date(elect_2010, Sys.Date(), "day") # Vector of days between general elections | |
| front_url <- "https://hansard.parliament.uk/commons" #Base URL for website + Commons | |
| day_urls <- paste0(front_url, "/", x) #Generate URLs for each day |
dbrby
/ snp21_conference_tmod.R
Created
September 13, 2021 16:55
Analysing party conference speeches
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| pacman::p_load(rvest, purrr, textclean, tidyverse, quanteda, | |
| quanteda.textplots, quanteda.textmodels, stm) | |
| pg <- read_html("https://www.snp.org/news/?_load_more=2") | |
| links <- pg %>% html_elements("a") %>% html_attr("href") %>% | |
| str_subset("address-to-snp21") %>% unique() | |
dbrby
/ senedd_rcv_ipe.R
Last active
September 14, 2021 19:54
Parse XML vote records from record.senedd, match to wikidata and estimate IPEs
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ## Estimate Ideal Points for Members of the Fifth Senedd from RCVs | |
| #install.packages("pacman") | |
| #install.packages('remotes') | |
| remotes::install_github('wmay/dwnominate') | |
| pacman::p_load(rvest, lubridate, tidyverse, readr, data.table, | |
| XML, pbapply, tidywikidatar, reshape2, pscl, | |
| wnominate) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| require(quanteda) | |
| require(readtext) | |
| require(tidyverse) | |
| require(quanteda.textmodels) | |
| require(quanteda.textplots) | |
| path_data <- system.file("/Users/danbraby/labour2021/", package = "readtext") | |
| # first save txt files from Github | |
| dat <- readtext("/Users/danbraby/labour2021/txt/*", |
dbrby
/ textmining4torybashing.R
Created
October 7, 2021 23:52
Average Words Per Sentence for Con21 Speeches
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # install.packages("pacman") | |
| pacman::p_load(rvest, tidyverse, quanteda) | |
| links <- c("https://www.conservatives.com/news/prime-minister-boris-johnson-speech-conference-2021", | |
| "https://www.conservatives.com/news/sajid-javids-speech-to-conservative-party-conference", | |
| "https://www.conservatives.com/news/taking-the-tough-decisions-to-cut-crime", | |
| "https://www.conservatives.com/news/we-will-make-sure-every-victim-sees-justice-done", | |
| "https://www.conservatives.com/news/rishi-sunak-speech-conference-2021", |