140.776 Statistical Computing class summaries

2026-09-08_notes.md

📚 Class Summary - Week 2: Literate Programming & Data Management

🎯 Main Topics Covered

📝 Introduction to Literate Programming

• 🔀 Documents combining code, output, and text/thoughts in one file
• 📊 Can create reports, presentations, websites for various audiences
• 🧩 Code chunks run sequentially with text explanations in between
• 📜 Historical context: Sweave (S + Weave) → R Markdown → Quarto

💻 Code Chunk Basics

• ✨ Start with three backticks, specify language (R, Python, etc.)
• 🏷️ Can name chunks and set options like:
  • eval: whether to run the code
  • echo: whether to show the code in output
• 📐 Two syntax styles: inline options vs. multi-line (newer, cleaner)

🔧 Tools Ecosystem

• 🐼 Pandoc: converts documents to various formats (HTML, PDF, Word, PowerPoint)
• ✍️ Markdown: syntax for formatting text (bold, lists, links, footnotes)
• 📘 R Markdown: R package for literate programming
• 🚀 Quarto: next-gen tool, supports multiple languages (R, Python, etc.)

🌟 Why Quarto?

• 🔄 More flexible for switching/combining programming languages
• 🎓 Future-proof for career beyond Hopkins
• 📁 File extension: .qmd

🛠️ Positron Tips & Shortcuts

• ⌨️ Cmd+Enter: Run code line
• 💬 Cmd+Shift+C: Toggle comment
• ➕ Cmd+Shift+I: Insert new code chunk
• 👁️ Render on Save: Auto-preview changes

📖 Reference Management

• 📦 Use citation("package_name") to get citation info
• 📄 BibTeX files (.bib) store citation data
• ✏️ knitr::write_bib() exports package citations
• 🔗 Inline citations: [@key] syntax
• 🎨 CSL files control citation formatting styles
• 📚 Zotero integration available in Positron

📂 File Paths & Data Management

• 🚫 Avoid setwd() - creates reproducibility issues
• ✅ Use here package for relative paths
• 🏠 here() finds project root automatically
• 🗃️ Useful functions: file.exists(), dir.create(), list.files(), file.remove()

📥 Downloading Course Data

• 🔗 Options: manual download, clone repository, or use provided R code
• ⏰ Download before Thursday for next class!

👥 Student Q&A Highlights

• ❓ E: What does echo do? → Controls whether code appears in output
• ❓ Question about Quarto vs R Markdown: Quarto recommended for flexibility with other languages
• ❓ J: When to start new code chunk? → When creating outputs, adding explanatory text, or after graphics

🎬 Fun References

• ⚽ FIFA World Cup mentioned (Go Mexico! 🇲🇽)
• 🧙 Wizard of Oz/Wicked analogy for here package - "follow the yellow brick road!"

2026-09-10_notes.md

📚 Class Summary: R Data Reading & Manipulation with dplyr

📖 Recap from Previous Class

• 📁 The here package helps write shareable code using relative paths
• 🔗 Works well with GitHub for easy project file sharing

📥 Reading Data in Base R

• 🔧 Key functions: read.table, read.csv, readLines, write.table, writeLines
• 📝 Base R naming inconsistency (camelCase vs periods) shows multiple developers
• 💡 read.csv works easily with clean CSV files - minimal options needed
• 💲 Access columns using dollar sign syntax: df$Team
• ⚠️ Avoid using full file paths - not shareable!

🔍 readLines vs read.csv

• 📄 readLines reads raw text without formatting (33 character strings)
• 🧹 read.csv automatically recognizes comma separators and formats data into tables

💾 Saving & Loading R Objects

• 📦 RData files: Store multiple objects, use save() and load()
• 📄 RDS files: Store single objects, use saveRDS() and readRDS()
• ✅ RDS allows renaming when loading; RData overwrites existing objects
• 💡 Use verbose = TRUE with load() to see what's being loaded

📚 Other Data Formats

• 🔄 haven: Read Stata, SPSS, SAS files
• 🌐 httr: Read web data
• 📊 readxl: Read Excel files
• ☁️ googlesheets4/googledrive: Access Google files
• 🕷️ rvest: Web scraping (use cautiously!)

💻 Memory Considerations

• 🧮 Calculate memory: rows × columns × 8 bytes (for numbers)
• 📏 Example: 1.5M rows × 120 columns = ~1.34 GB RAM needed
• 🔧 Workaround: Read large files in chunks

📖 readr Package (Tidyverse)

• 🏷️ Uniform naming: read_csv, read_tsv, read_delim
• 📊 Shows column types automatically (chr, dbl, etc.)
• 🎯 Returns tibbles instead of data frames
• ✨ Only prints first 10 rows - no accidental flooding!
• ⏭️ skip parameter ignores header comment lines
• 💬 comment parameter ignores lines starting with specified character
• 🗜️ Can read compressed files directly

🗂️ Tibbles vs Data Frames

• 📋 Tibbles show dimensions, column types, and limited rows when printing
• 🚫 Tibbles don't allow row names (more predictable)
• ✅ Tibbles allow non-standard column names (spaces, numbers)
• 👀 Use View() for Excel-like exploration in RStudio/Positron

🛠️ dplyr Package - Grammar of Data Manipulation

• 👨‍💻 Created by Hadley Wickham (one of his first famous packages!)
• 🗣️ Functions named as English verbs for readable code

🔑 Core dplyr Verbs

Verb	Purpose
select	Choose columns/variables
filter	Keep rows meeting conditions
arrange	Reorder rows
rename	Change column names
mutate	Create/transform columns
group_by	Group observations

🔗 The Pipe Operator

• 📝 Tidyverse: %>% | Base R: |>
• ➡️ Connects operations left-to-right
• 🔄 All dplyr functions: data in first, data frame out

🧹 Tidy Data Principles

• 📊 Variables = columns, Observations = rows
• 📦 Each cell = single measurement
• ⚠️ Not the same as "pretty" Excel formatting!

🔧 dplyr Examples

• ✂️ Select: select(city:dptp) or drop with -
• 🔍 Filter: Use ==, !=, >, <, %in%, is.na()
• 📶 Arrange: arrange(date) or arrange(desc(date))
• 🏷️ Rename: rename(dewpoint = dptp)
• ➕ Mutate: Create new columns from existing data

⚠️ Best Practices

• 🔢 Avoid accessing columns by position number (causes silent errors!)
• 🏷️ Use descriptive variable names
• ❌ Use !is.na() to filter out missing values
• 🧮 Use na.rm = TRUE when computing means with missing data

📅 Homework Reminders

• 📝 Project Zero: Due Sunday - post on CoursePlus forum
• 🚀 Project One: Start early! Due September 20th
• 🏢 Attend office hours with TAs or instructor for help

2026-09-15_notes.md

📚 Class Summary - Week 3: Data Visualization & Tidyverse

🌐 Project Zero Website Showcase

• ~15 students completed optional website project and shared on Course Plus
• Various templates used: Trestles (most popular), and others
• Highlights included:
  • 📧 Contact links (email, GitHub, LinkedIn)
  • 🎨 Custom colors, icons, and emojis
  • 📸 Profile photos (circular, oval, square formats)
  • 🔗 Fun facts, hobbies, resources sections
  • ⭐ One student included a favicon (small browser tab icon)

💡 Website Tips

• 🔍 Google yourself to see what others find about you
• 🔗 Include links to things that are hard to Google (advisors, past employers)
• 📱 Consider separating personal vs. professional social media
• ✏️ Keep websites updated as information changes
• 🎯 Make yourself memorable to recruiters

📊 Dplyr Functions Continued

🔀 Group By & Summarize

• group_by(): Splits data into groups (e.g., by year)
• summarize(): Computes summary statistics per group (mean, max, median, etc.)
• Always used together! 🤝

🔗 Pipe Operator (%>%)

• Passes output from left side to first argument on right side
• Makes code more readable (read top to bottom instead of inside-out)
• Very popular - other languages try to emulate it!

🍕 Slice Functions

• slice_sample(): Random rows (like grabbing a random pizza slice!)
• slice_head(): First n rows
• slice_tail(): Last n rows

🧹 Tidy Data & TidyR

📐 Tidy Data Definition

• Each variable = column
• Each observation = row
• Each cell = single measurement

🔄 Pivot Functions

• pivot_longer(): Wide → Long format (makes data tidy!)
• pivot_wider(): Long → Wide format (good for presentations)
• 📏 Check row counts to verify transformations worked correctly

📦 Tidyverse Package

• Bundles related packages: dplyr, tidyr, ggplot2, magrittr, etc.
• One library(tidyverse) loads them all!

🔗 Joining Data (Relational Data)

🔑 Key Concepts

• Primary key: Uniquely identifies observations in query table
• Foreign key: Matching variable in target table
• Keys link tables together!

🎯 Join Types (Venn Diagram Style)

• inner_join(): Keep only matching observations (intersection) ⭕
• left_join(): Keep everything in X, add matching from Y ◀️
• right_join(): Keep everything in Y, add matching from X ▶️
• full_join(): Keep everything from both (union) ⬛

⚠️ Missing Data (NAs)

• Joins can create NAs when observations don't match
• Studying patterns of missing data is its own field!

📈 Visualization Preview

📚 Recommended Resources

• R Graphics Cookbook (recommend buying print version! 📖)
• Posit cheat sheets (great for reference)
• Online resources searchable but harder to browse visually

🎨 Coming Thursday

• Actually making visualizations with ggplot2! 🎉

2026-09-17_notes.md

📊 Class Summary: Data Visualization with ggplot2

🎨 Introduction to ggplot2

• 📚 Learning to make plots using ggplot2, part of the Tidyverse
• 👨‍💻 Developed by Hadley Wickham, who also wrote "The Grammar of Graphics" book
• 📖 R Graphics Cookbook recommended for browsing visualization examples and inspiration

🖼️ Grammar of Graphics Concept

• 🎭 Think like an artist: start with a blank canvas (background)
• 📋 First argument is always a data frame (tidy format assumed)
• 🗺️ Aesthetic mappings define how variables map to x-axis, y-axis, color, size, shape
• ➕ ggplot2 uses plus signs to combine functions (inspired the pipe syntax later)

🔧 Key Components

• Data 📊 - tidy data frame
• Aesthetics 🎯 - mapping variables to visual properties
• Geoms 📍 - how to display data (points, lines, etc.)
• Facets 📑 - split graphs by groups (rows/columns)
• Statistics 📈 - quantiles, binning
• Scales ⚖️ - transform x/y axis visualization
• Coordinates 🗺️ - Cartesian, polar, etc.

🐧 Practice Datasets

• 🏥 MAACS dataset - Mouse Allergen and Asthma Cohort Study (simulated data for privacy)
• 🐧 Palmer Penguins - modern alternative to IRIS dataset (controversy with IRIS author)

📝 Key Functions Demonstrated

• ggplot() - creates the base object
• geom_point() - scatter plots
• geom_smooth() - adds trend lines (method = "lm" for linear regression)
• facet_grid() - creates multi-panel plots
• labs() - customize labels, titles, captions
• aes() - aesthetic mappings (can be global or layer-specific)

🎭 Customization Options

• Alpha blending 🌫️ - control transparency for overlapping points
• Point size/color 🔵 - constant values or mapped to variables
• Line width/type 📏 - customize smooth lines
• SE = FALSE ❌ - remove confidence bands

🎨 Themes

• Built-in: theme_gray(), theme_bw(), theme_dark()
• 📰 BBC Style - bbplot package for news-style graphics
• 🎬 Theme Park package - Barbie, Oppenheimer, Harry Potter, Game of Thrones themes
• 📊 ggthemes package - Wall Street Journal, Economist, Stata styles
• 👨‍🎨 Example: Matthew B. Jane created personal brand with consistent pink/blue color scheme

🖱️ Interactive Graphics

• 📦 Plotly package with ggplotly() function
• 🔍 Enables zoom, hover tooltips, point selection
• 💻 Works great for HTML outputs/websites

♿ Accessibility

• 🎨 colorblindr package by Claus Wilke
• cvd_grid() function simulates how graphs appear with:
  • Deuteranomaly
  • Protanomaly
  • Tritanomaly
  • Desaturated colors

📋 Upcoming

• 🤖 Tuesday: Guest lecture on Posit Assistant (AI coding tool)
• 💡 Recommended: Sign up for Posit AI free trial or get GitHub Education Pack approved
• 📝 Project 1 requires making 7 modifications to improve a graph

2026-09-22_notes.md

📚 Class Summary - Week 4: AI Coding Tools & R Basics

🤖 Introduction to AI Coding with Posit Assistant

🎯 Guest Speaker: Nick Eagles (LIBD, 8 years experience)

• Demonstrated using Posit Assistant in Positron for AI-assisted coding
• Showed how to solve Project 1 using AI tools

📊 Choosing AI Models

• Top tier: Claude Opus, GPT-6 Astra (most expensive)
• Mid tier: Claude Sonnet, GPT-5.4 (recommended for most tasks)
• Basic tier: Claude Haiku (simple tasks like basic plots)
• ⚠️ More powerful models = higher cost (up to 50x more expensive!)

💰 Understanding Costs & Tokens

• LLMs operate on tokens (words/punctuation broken into pieces)
• Input tokens are cheaper than output tokens
• 📈 Longer chats = higher costs (entire chat sent with each message)
• 🚫 Don't switch models mid-chat (breaks caching, increases costs)

📁 Context Files (AGENTS.MD)

• Place at project root for persistent AI knowledge
• Auto-read by AI in every chat
• Include: project goals, coding standards, file locations
• Keep concise - relevant info only

💻 Live Demo Highlights

✅ What Worked

• Generated hundreds of lines of code in ~10 seconds
• Created histograms, violin plots, and data visualizations
• Provided reasonable explanations for coding choices

⚠️ Common Issues

• AI created unnecessary new files instead of editing existing ones
• Included NA values when not wanted
• Used advanced functions not yet covered in course (e.g., fct_relevel)
• Required validation - code may run but not be correct

🔧 Helpful Features

• Fix/Explain buttons for console errors
• Git integration to track AI changes (green = new, red = deleted)
• 💡 Tip: Commit before using AI to easily revert unwanted changes

📐 R Basics Review

🧱 Vector Types

• Atomic vectors: logical, integer, double, complex, character, raw
• Lists: can contain different types/lengths (data frames are special lists)

🔢 Numbers in R

• Integer: whole numbers (use 4L to force integer storage)
• Double: decimals (default for numbers, ~15 digit precision)
• ⚠️ Doubles are approximations: sqrt(2)^2 ≠ exactly 2

🔄 Type Coercion

• R automatically converts mixed types in vectors
• Hierarchy: character > double > integer > logical
• TRUE → 1, FALSE → 0 (useful for mean() on logical vectors!)

💡 Pro Tips

• ✅ Use TRUE/FALSE not T/F (T/F can be overwritten!)
• Use typeof() and length() to inspect objects
• Incomplete expressions show + prompt (not >)

📚 Resources

• Follow Simon Couch and Sarah Alman on BlueSky/LinkedIn for Posit AI updates
• LIBD R Stats Club recordings available for additional learning

2026-09-24_notes.md

📚 Class Summary: R Programming - Control Structures & Data Types

🔄 Coercion in R

• ✨ Explicit coercion uses as.* functions (e.g., as.numeric, as.logical, as.character) to intentionally convert between vector types
• ⚠️ When coercion fails, R introduces NA values with a warning message
• 💡 Using tools like Positron or the colorout package helps make warnings more visible (shown in red)

📊 R Object Types Review

• 🧱 Matrices: Vectors with a dim attribute; created with matrix() or by setting dim() on a vector
• 📋 Data frames: Store tabular data with different column types; created with data.frame()
• 🏷️ Factors: Categorical data stored as integers with character level labels
  • ⚠️ Can be confusing - math operations work on integer indices, not displayed values!
  • 📝 Before R 4.0, data was often auto-converted to factors (now defaults to character)
• ❓ Missing values: Use is.na() for NA values; is.nan() specifically for NaN

🔀 Control Flow Structures

🎯 If/Else Statements

• 📝 Syntax: if (condition) { code } else { code }
• 🔗 Can chain with else if for multiple conditions
• 💻 Useful in Quarto/R Markdown with eval and echo chunk options for dynamic reports

🔁 For Loops

• 📝 Syntax: for (i in sequence) { code }
• 🛠️ seq_along() helper function creates integer sequences matching object length
• 📊 Common for iterating over data columns or matrix elements
• ✅ Use i, j naming convention for rows/columns

⏳ While & Repeat Loops

• 🔄 While: Repeats while condition is TRUE; update condition inside loop
• ♾️ Repeat: Infinite loop requiring break to exit
• ⏭️ Next: Skips current iteration

📌 Key Reminders

• 📅 Project 2 due Sunday next week
• 💻 Next week's classes will be on Zoom
• ❓ Use CoursePlus Discussion Forum for questions

2026-09-29_notes.md

📚 Class Summary: Functions & Loop Functions in R

🔧 Why Learn Functions?

• 📊 Repeat same steps (summaries, graphs) with different inputs
• 🔄 Avoid copy-pasting code - make it reusable
• 📝 Document options/parameters for yourself and others
• 🤝 Share code publicly or internally
• ✅ Core activity of R programmers

---

🏗️ Function Structure Basics

• 📦 Use function() keyword with parentheses for parameters
• 📎 Code goes inside curly brackets {}
• 💾 Save function to an object (e.g., f <- function() {})
• 🏷️ Functions are a type of object (class = "function")
• 👋 Classic "Hello World" example demonstrated

---

📤 Function Arguments & Returns

• 🎛️ Arguments/Parameters: Options passed to functions (e.g., num)
• 🔙 Return values: Use return() or last expression automatically returns
• ⚙️ Default values: Set with = sign (e.g., num = 1) - makes functions user-friendly
• 🔍 Use formals() to see function arguments and defaults

---

🎯 Argument Matching

• 📍 Positional matching: Arguments assigned by order (not recommended - hard to read)
• 🏷️ Named matching: Specify argument names explicitly (recommended!)
• ⚠️ Mixing both can be confusing - avoid when possible
• 💡 Best practice: Only name arguments with non-default values

---

😴 Lazy Evaluation

• R executes code line by line inside functions
• Only checks for needed objects when that line runs
• Can cause confusing partial outputs before errors

---

📌 The Ellipsis Argument (...)

• 🔄 Allows functions to accept unlimited additional arguments
• 🎨 Useful for wrapper functions (e.g., plotting)
• ⚠️ Arguments after ... must use named matching

---

✍️ Function Naming Best Practices

• 🏃 Use verbs for function names, nouns for arguments
• 📏 Balance: not too short (f), not too long
• 🐍 Be consistent with style (snake_case vs camelCase)
• 🔤 Use common prefixes for autocomplete benefits
• 🚫 Never override existing R functions (mean, sum, c, etc.)
• 💬 Add comments inside functions to explain code

---

🔁 Loop Functions & Vectorization

• ➕ R has built-in vector arithmetic (element-by-element operations)
• 📐 sqrt(), +, -, *, / work on entire vectors
• 🔄 Apply family (base R): lapply, sapply, mapply, etc.
• 🐱 purrr package (Tidyverse): More predictable and consistent

---

🐱 The purrr Package

• 🗺️ map() functions apply a function to each element
• 🎯 Specific output types: map_int(), map_dbl(), map_chr()
• 2️⃣ map2() for two input vectors
• 📝 Formula syntax with ~ and .x, .y shortcuts
• 🧁 Cupcake frosting analogy illustrated the concept!

---

📦 Sharing Functions (Optional/Future Reference)

• 📄 Roxygen2: Document functions with special comments
• 🧪 testthat: Write tests for your functions
• 📦 Can create R packages to share functions publicly

---

❓ Q&A Highlights

• ✅ OK to mix base R and Tidyverse - use what you know!
• 📅 Project 2 due Sunday - includes function-writing example
• 💬 Use CoursePlus Discussion Forum for questions

---

📅 Next Class

• 🐛 Debugging code in R
• ⚠️ Error handling in R
• 🖥️ Thursday: Remote | Next Tuesday: In-person

2026-10-01_notes.md

Class Summary: R Loop Functions & Debugging 📚

Loop Functions in R 🔄

Base R lapply (List Apply)

• 📋 Takes a list as input and applies the same function to all elements
• 🔧 Most versatile/flexible apply function since lists can contain different object types
• 📤 Returns a list as output
• 💡 Can use with data frames and triangular data (elements of different lengths)

Using Ellipsis (...) with lapply

• ➕ Additional arguments after the function are passed to the applied function
• 🎯 Useful for exploring functions across a range of input parameters while keeping others constant

purrr Package Equivalents

• 🗺️ map() = equivalent to lapply() (returns list)
• 🔢 map_dbl() = returns numeric/double vector
• 📝 map_chr() = returns character vector
• 🔣 map_int() = returns integer vector

Anonymous Functions with lapply

• 🏥 Example: Extracting baseline data from multiple hospital datasets
• 💾 Can save functions separately for testing/documentation instead of using anonymous functions

Other Apply Functions

• 📊 sapply = Simple Apply (simplifies output)
• 📈 tapply = Table Apply
• 🔲 apply = For matrices/data frames (across rows or columns)

split() Function

• ✂️ Divides data into a list based on a grouping variable
• 🤝 Combined with loop functions = similar to dplyr's group_by() + summarize()
• 📈 More flexible for complex input data than dplyr

Vectorizing Functions ⚡

• 🔧 Vectorize() converts single-value functions to accept vector inputs
• 🚀 Useful for expanding existing functions to loop across multiple inputs

Parallelization Options 🖥️

• ⏱️ Useful when computations take long (e.g., 100 datasets × 10 min each)
• 📦 furrr package: Parallelized purrr (just add future_ prefix)
• 📦 BiocParallel: bplapply() instead of lapply()

---

Debugging Code in R 🐛

Visibility Tools

• 🎨 colorout package: Color-codes errors (red), warnings, and different data types
• 📝 Add to .Rprofile for automatic loading via usethis::edit_r_profile()

Finding Help

• 🔍 When Googling errors: Remove object names, add "R" to search
• 🤖 Use LLM/AI "explain" features in Positron

Making Errors Reproducible

• 📦 reprex package: Creates reproducible examples
• ⚙️ Tip: Disable "Restore .RData" in RStudio settings

Testing with testthat

• ✅ expect_equal() - test expected outputs
• ❌ expect_error() - test that errors occur when expected

---

Error Handling & Generation 🚨

Three Types of Conditions

Type	Purpose	Function
📨 Message	Informational updates	message()
⚠️ Warning	Something unexpected but continues	warning()
🛑 Error	Fatal problem, stops execution	stop()

rlang Equivalents (Tidyverse style)

• stop() → abort()
• warning() → warn()
• message() → inform()

CLI Package for Fancy Messages

• 🎨 cli_abort() provides structured, colorful error messages with hints and tips
• 😊 praise package: Random positive messages for encouragement!

Best Practices

• ✅ Use stopifnot() for quick input validation
• ✅ Use rlang::arg_match() for argument matching with helpful typo suggestions
• ✅ Check inputs early to avoid wasting user time
• ✅ Use call = FALSE in stop() for cleaner error messages
• 🔍 traceback() or rlang::last_error() shows the function call stack

When to Use What

• 🛑 Error: Invalid inputs, conditions that will definitely fail
• ⚠️ Warning: Unexpected situations where code might still work
• 📨 Message: Progress updates, informational output

---

Key Takeaways 🎯

• 🔄 Loop functions eliminate repetitive code
• 🏥 Great for analyzing data across multiple sources (e.g., hospitals)
• 🎨 colorout helps spot errors visually
• 📝 Write user-friendly error messages for yourself and others
• 🧪 Test your functions with testthat

2026-10-06_notes.md

📚 Class Transcript Summary

🎉 Welcome Back & Course Updates

• Welcome back to in-person classes after problem set 2 on writing functions
• Starting the final 3 weeks ("home stretch") of the course
• 📝 Request for students to share frustrating R experiences on CoursePlus discussion forum to help adapt remaining lessons

🔧 Error Handling in R (Continued from Previous Class)

Try-Catch Function

• 🎯 Purpose: Try to execute code and catch what happens (errors or warnings)
• Can convert warnings into errors or vice versa
• Useful for handling code from other people's functions

📋 Practical Example: Download Retry Function

• Uses tryCatch inside a while loop
• Retries downloading files up to 3 times with random delays (2-5 seconds)
• Helpful when web connections might fail randomly

⚠️ Performance Warning

• tryCatch makes code significantly slower (~15x slower in example)
• Fine for infrequent operations (like downloads)
• Avoid in code that runs thousands of times
• 📊 Use microbenchmark package to compare code execution speeds

💡 QMD Tip

• There's a code option to show errors in Quarto documents without crashing (instructor will post example later)

📅 Working with Dates and Times (Main Topic)

🌟 Why This Matters

• Needed for Project 3 (Beyoncé & Taylor Swift album release dates)
• Very common in longitudinal/patient data analysis

📦 Key Package: lubridate

• Part of tidyverse but needs separate loading
• Created by Hadley Wickham
• Handles time zones, leap years, leap seconds automatically

🔢 How Dates/Times Are Stored

• Dates: Number of days since January 1, 1970
• Times: Number of seconds since January 1, 1970 (UTC)
• Called "epoch time" or "POSIX time" (from Unix)

📝 Three Main Object Types

1. Date - just month/day/year
2. Time - just hour/minute/second (rarely used alone)
3. DateTime - both combined

🛠️ Creating Date Objects

• today() - current date
• now() - current datetime
• YMD family functions: ymd(), mdy(), dmy() - letter order matches input format
• Very flexible - recognizes multiple formats automatically!

🛠️ Creating DateTime Objects

• Add _hms, _hm, or _h suffix for time components
• Example: ymd_hms() for year-month-day hour:minute:second
• tz argument for time zones (default: UTC)

🔄 Combining Columns into Dates

• make_date(year, month, day) - when data is in separate columns
• make_datetime(year, month, day, hour, min, sec) - includes time
• Use with mutate() from dplyr

🌍 Time Zones

• Default is UTC (Coordinated Universal Time)
• Leave tz="" to use your computer's local time zone
• R recognizes daylight saving time automatically
• Change display timezone with attr(x, "tzone") without changing underlying data

➕➖ Math with Dates/Times

• ✅ Subtraction works (gives difference in days/seconds)
• ✅ Comparison operators work (<, >, ==)
• ❌ Adding two datetime objects doesn't work
• ✅ Can add seconds to datetime: x + (3 * 60 * 60) adds 3 hours
• ✅ Can add days to dates: date + 1
• 🔄 R handles leap years and leap seconds automatically

📤 Extracting Components

• year(), month(), day(), weekdays()
• hour(), minute(), second(), week()
• Great for grouping/summarizing data!

📊 Plotting with Dates

• ggplot2 automatically handles date axes
• Intelligently adjusts labels (shows months vs. hours based on data range)
• Use filter() with date functions: filter(month(begin) == 6)

🎨 Bonus Tips

• 📦 ggview package: Preview how graphs look at specific dimensions
• 📦 showtext package: Resize text for different resolutions
• R community on BlueSky/social media is great for discovering new packages

📋 Reminders

• Share R frustrations on discussion forum
• ⏰ Today's deadline: Submit info for GitHub Education Pro accounts

2026-10-08_notes.md

📚 Class Summary: Regular Expressions in R

🎯 Introduction to Regular Expressions (Regex)

• 📝 Regular expressions help define specific patterns to search for in text
• 🔤 Useful for extracting information from human language/text data
• 💡 Examples: searching syllabi for programming languages, extracting recipe ingredients
• ✂️ Can also be used to search and replace text (like Word/Google Docs find & replace)

🔧 Basic Components

• 📌 Two strings needed: pattern (what to search for) and text (where to search)
• 🔠 Strings can be defined with double quotes (") or single quotes (')
• ⚠️ Press Escape if you forget to close a string

🛠️ Key R Functions

• 📊 grepl() - returns TRUE/FALSE logical vector for pattern matches
• 📥 Arguments: pattern (length 1) and x (any length character vector)

🎭 Meta Characters

🔵 The Period (.)

• Represents any single character (except newline)
• Acts like a "joker card" in patterns

🔁 Repetition Modifiers

• ➕ + = one or more times
• ✳️ * = zero or more times
• ❓ ? = zero or one time
• 📐 {n} = exactly n times
• 📐 {n,} = n or more times
• 📐 {,m} = at most m times
• 📐 {n,m} = between n and m times

📦 Capture Groups

• 🔲 Use parentheses () to group patterns for repetition
• 🪆 Can be nested for complex patterns

🔤 Character Sets

• 📋 \w = word characters (letters, digits, underscore)
• 🔢 \d = digits (0-9)
• ⬜ \s = whitespace (spaces, tabs, newlines)
• 🔄 Capital versions = negation (\W, \D, \S)
• ⚡ Must use double backslash (\\d) in R

🎨 Custom Character Sets

• 🔲 Use square brackets [] to define custom sets
• 📝 Example: [aeiou] for vowels
• ➖ Use dash for ranges: [a-m], [0-9]
• 🚫 Use ^ inside brackets for negation: [^aeiou]

📍 Anchors

• 🎬 ^ = beginning of string (outside brackets)
• 🏁 $ = end of string

⚡ OR Operator

• 🔀 Use pipe | for alternatives: north|south

💻 Practice Example

• 🗺️ Used state.name dataset (50 US states)
• 🔍 Found states starting AND ending with vowels using complex regex pattern