Skip to content

Instantly share code, notes, and snippets.

Title Eval_1 Eval_2 Category Rating_1 Confidence_range_1 Comments_1 Rating_2 Confidence_range_2 Comments_2
The Comparative Impact of Cash Transfers and a Psychotherapy Program on Psychological and Economic Well-being anonymous anonymous Overall assessment 90 3 75 (65, 85)
Advancing knowledge and practice 90 2 70 (60, 90) It provides useful evidence on the (in)effectiveness of CBT for general populations and was well-designed to investigate other important questions.
Methods: Justification, reasonableness, validity, robustness 90 3 90 (85, 95) A well-executed RCT with effort to avoid bias.
Logic & communication 80 4 75 (70, 90) Generally good, some claims could be better supported.
Open, collaborative, replicable 70 5 50 (40, 80) Data and code arenÕt available but I think this is standard for unpublished papers so I downweight this category. ThereÕs some small discrepancies between the numbers reported in section III.B and the actual numbers in Table 1. Pre-analysis plan is av
We can make this file beautiful and searchable if this error is corrected: Unclosed quoted field in line 6.
Title,Eval_1,Eval_2,Category,Rating_1,Confidence_range_1,Comments_1,Rating_2,Confidence_range_2,Comments_2
Mental Health Therapy As a Core Strategy For Increasing Human Capital: Evidence From Ghana,Anonymous,Anonymous,Overall assessment,75,4,,75,"(70, 84)",
,,,Advancing knowledge and practice,65,4,,60,"(55, 65)",
,,,"Methods: Justification, reasonableness, validity, robustness",60,3,,90,"(82, 94)",
,,,Logic & communication,75,3,,70,"(62, 82)","I wish these categories were separated - would rank it high for logic but lower for communication. Even though this is an extremely well written (clear and easy to follow) paper, I struggled with some of the framing and messaging (i.e. higher-level communication)."
,,,"Open, collaborative, replicable",50,3,,90,"(80, 95)","Data and code are provided alongside the published paper (https://www.openicpsr.org/openicpsr/project/164481/version/V1/view) I have not tried to reproduce any of the analyses as I do not have access to Stata, hence wider CIs. Readme file is detailed a
We can make this file beautiful and searchable if this error is corrected: It looks like row 3 should actually have 15 columns, instead of 14 in line 2.
Category_abbrev,Rating_1,Confidence_range_1,Rating_2,Confidence_range_2,Rating_3,Confidence_range_3,Comments_1,Comments_2,Comments_3,Category_described,Title,Eval_1,Eval_2,Eval_3
Overall,80,"(70, 90)",80,4,79,"(59,94)","Excellent as an overview, and important for global health, but unfortunately somewhat disappointing about ways to address global catastrophic risks. [EditorÕs note: the evaluator has explained that they have not based their assessment on this point; we do not intend Ê'Relevance to Global Priorities"" to factor into the overall assessment)",,,Overall assessment,Advance Market Commitments: Insights from Theory and Experience,David Manheim,Dan Tortorice,Joel Tan
Adv. Knowl.,25,"(20,40)",90,5,90,"(70, 100)","This was a review, not intended to directly advance knowledge. To the extent that it claims additional usefulness of AMCs, it seems not to have addressed other options.",,,Advancing knowledge and practice,Advance Market Commitments: Insights from Theory and Experience,David Manheim,Dan Tortori
We can make this file beautiful and searchable if this error is corrected: It looks like row 6 should actually have 11 columns, instead of 9 in line 5.
Category_abbrev,Rating_1,Confidence_range_1,Rating_2,Confidence_range_2,Comments_1,Comments_2,Category_described,Title,Eval_1,Eval_2
Overall,80,"(70, 90)",50,4,,,Overall assessment,Artificial Intelligence and Economic Growth,Seth Benzell,Philip Trammell
Adv. Knowl.,75,"(65, 85)",90,5,,,Advancing knowledge and practice,Artificial Intelligence and Economic Growth,Seth Benzell,Philip Trammell
Methods,80,"(75, 85)",80,4,,"Somewhat scattered, and in that sense less robust","Methods: Justification, reasonableness, validity, robustness",Artificial Intelligence and Economic Growth,Seth Benzell,Philip Trammell
Logic & Comm.,70,"(60, 80)",80,4,,"An awkward combination of intensive focus on some things and selective breadth in others. Also, unusually many typos and minor errors. On the other hand, logical and very clearly written.",Logic & communication,Artificial Intelligence and Economic Growth,Seth Benzell,Philip Trammell
Collab.,95,"(90, 100)",90,3,,,"Open, collaborative, replicable",Artificial Intelligence and Econ
@daaronr
daaronr / eval_sample.csv
Last active March 16, 2023 02:28
ratings_and_prediction_metrics
paper name here John Smith Anonymous Jane Doe
Category Rating (0-100) 90% CI (0-100) Comments (footnote) Rating (0-100) Confidence: * High = 5, Low = 0 Comments Rating (0-100) 90% CI (0-100) Comments
Overall assessment 50 (40, 65) 50 4 79 59-94
Advancing knowledge and practice 25 (20,40) 90 5 90 70-100
Methods: Justification, reasonableness, validity, robustness 95 (85,97.5) 80 4 70 50-90
Logic & communication 75 (60,90) 80 4 70 50-90
Open, collaborative, replicable N/A N/A 90 3 50 30-70
Engaging with real-world, impact quantification; practice, realism, and relevance 90 70-100
```{r}
library(readr)
gifts_2022_12_15_2017 <- read_csv("~/Downloads/gifts_2022-12-15_2017.csv")
Downloaded from link in https://yieldgiving.com/gifts/?sorting=az
```
```{r}
# Load the stringr package
library(stringr)
@daaronr
daaronr / bookdown_to_quarto.md
Last active December 22, 2022 20:00
going from Bookdown (rethinkpriorities package or Reinstein's bookdowns) to Quarto books

Bookdown to Quarto -- steps and considerations

#rstats I think I'll switch to Quarto. It seems it can do all Bookdown can do & more, with better support.

- Anyone know tools/templates for converting projects from Rmd/Bookdown to Quarto?
- Am I losing something from making the switch?

@xieyihui

— 𝙳𝚊𝚟𝚒𝚍 𝚁𝚎𝚒𝚗𝚜𝚝𝚎𝚒𝚗 (@GivingTools) March 30, 2022

At Rethink we set up this template and tools.

For much of my work I was using the Bookdown style and template I worked on here

But I think I am (and we are) moving to Quarto. ... Quarto pages and Quart

Git/Github Question I keep coming back to

  • I need to maintain 2 versions of a page, site, or report, for different audiences. E.g., one ‘public’ and one ‘private within group’

  • I keep each of these on distinct branches in the same repository

  • Alternatively I could make these separate repositories, one fork in the other, although that feels more bulky

@daaronr
daaronr / making_gh_repo_with_cli.md
Created January 11, 2022 22:38
Suprisingly easy to set up GH the CLI and make a repo ... Just some bash script and CLI responses

Setting up the Github terminal access -- 'gh cli' and setting up a repo

I waited a long time to try this, but it proved surprisingly easy

Install GitHub CLI, authenticate

In the bash terminal ... if you have brew installed on a Mac

Discussion and annotation of Osborne and Rubinstein "Models in Microeconomic Theory"

(Discussion between David Reinstein and Britt Li, possibly others)

Chapter 1

1.1 Preferences