benmarwick

# This script will take a folder fill of qmd files and try to render each qmd file.
# If one qmd fails to render, the script will continue on to the next one. The
# results of all the attempts to render are collected in a data frame so we can 
# easily inspect and find the files that failed to render We don't have to 
# manually render individual files.

# How to use

# 1. Download all submissions from Canvas (go to assignment page, look for the 
# download button near the speedgrader button), when it arrives on your

benmarwick

# How many articles on the list in Sept 2021?

# First, run some lines from archaepaperswithcode.R to create repo,
# then: 

## Coerce commits to a data.frame
df <- as.data.frame(repo) 

# filter rows of commits from Sept 2021

benmarwick

# JB says she searched for "archaeolog*" and "archeolog*", these return 257 results for me, 
# much less than the 1002 we get from searching the archaeology 'keyword', e.g.
# https://www.pnas.org/action/doSearch?Concept=500376&Concept=500375&startPage=0&sortBy=Earliest
 
# in any case, let's start with "archaeolog*" and "archeolog*", I've copied the URL of the search
# results page and edited the URL to return 500 items on the first page, so we can get all results without
# having to scrape multiple pages of results, just to simplify the process

library(tidyverse)

benmarwick

library(tidyverse)
library(textreuse)

# one row per student, to get the data, go to canvas -> quiz -> 'quiz stats' -> 'student analysis'
cnvs <- read_csv("quiz-responses-downloaded-from-canvas.csv")

# select only the column with the text we want to compare
cnvs_q5 <- 
cnvs %>% 
  select( q5 = contains("peers' reflections")) 

benmarwick

library(tidyverse)
library(sf)
library(googlesheets4)

# get our data from google sheets, we need to:
# - 'publish to web'
# - adjust sharing settings to share with anyone
# I've done these, now, so it should just work:
my_key <- "1xUqRGnb9kwBi128cERiHkV5w4uREwl-mqAVf0jmQhhg"

benmarwick

# also at https://gist.github.com/benmarwick/f11ae49ab9afde0071b133012ff76cbc

ctv <- "https://raw.githubusercontent.com/benmarwick/ctv-archaeology/master/README.md"

library(tidyverse)
library(glue)

archy_ctv_readme <- readLines(ctv)

# get just the articles

benmarwick

I used the Python library GetOldTweets3 to get the tweets because the rtweet package cannot get tweets older than 6-9 days. Details about this Python library are here: https://github.com/Mottl/GetOldTweets3

I used this line in the shell to get tweets using the #viralarchive hashtag: 

```{bash, engine.opts="-l", eval = F}
GetOldTweets3 --querysearch 'viralarchive' --maxtweets 10000
```

benmarwick

# How to save a ggplot as a JPG file with a 
# specific dpi and dimensions, for example,
# because a publisher requires it

library(ggplot2)

p <-
  ggplot(mtcars) +
  aes(mpg, 
      disp) +

benmarwick

library(tidyverse)
library(rvest)

# what quarter and year do we want to check for the availability of DSM courses?
qrt_year <- "AUT2020"

# this is the URL to our canonical list of DSM courses
webpage <- "https://www.washington.edu/uaa/advising/single-pages/data-science-minor/"

benmarwick

# from https://github.com/cboettig/compendium/blob/master/.github/workflows/main.yml
on: [push]

name: render all R Markdown documents 

jobs:
  render:
    name: render all R Markdown documents 
    runs-on: macOS-latest
    steps: