dantonnoriega

# SETUP -----------------
library(httr2)
api_base_url = "https://api-prod.etf.com/private"
hdrs <- list(
  `x-limit` = 10000,
  `User-Agent` = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.3.1 Safari/605.1.15',
  `Origin` = 'https://www.etf.com',
  `Referer` = 'https://www.etf.com/',
  `Accept` = '*/*',

dantonnoriega

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import statsmodels.api as sm

# Set random seed for reproducibility
np.random.seed(42)

# Generate synthetic data
n_obs = 100

dantonnoriega

Quick Way to Estimate Taxes Owed on Capital Gains (E*Trade)

This short explainer goes through

1. where to download your Gains and Losses sheet on one E*Trade account.
2. how to quickly estimate taxes owned.

---

(1) Downloading Gains & Losses Sheet (G&L_collapsed.xlsx) from E*Trade

dantonnoriega

Helpful Resources

• Where I learned about fc -p (read in and replace) and fc -A (Append)
  • https://unix.stackexchange.com/questions/391181/freezing-the-history-file-temporarily

WARNING: using fc -p will replace your current history file with a blank one. Tread carefully.

GOAL: merge (or concat or join or append) a set of old zsh_history files into a single one.

Imagine you have 3 zsh history files, with timestamps from oldest to newest: zsh_history0, zsh_history1, zsh_history2.

dantonnoriega

Databricks - JDBC

This is the preferred method if you're collecting small objects from the Spark Cluster because it is otherwise easy to set up, doesn't appear to cause many "out of memory" errors, and requires fewer dependencies.

Setup

1. Install and configure Java.
   • Install Java 8

brew tap AdoptOpenJDK/openjdk

dantonnoriega

# The CONNECT_SERVER URL must have a trailing slash.
connectServer <- "https://some-rsc-endpont.com"
connectAPIKey <- Sys.getenv("RSC_API_KEY")

# the content GUID of interest
content_guid_3pc <- "0a0b1c2d-1234-4321-a1c2-5678aaa9d9bb" # I made this one up

resp <- httr::GET(
  paste0(connectServer, "__api__/v1/instrumentation/shiny/usage?limit=100"),
  httr::add_headers(Authorization = paste("Key", connectAPIKey)),

dantonnoriega

# The CONNECT_SERVER URL must have a trailing slash.
connectServer <- "https://some-rsc-url.com/"
# Save RSC API key to ~/.Renviron as RSC_API_KEY
# - https://docs.rstudio.com/connect/1.7.4/user/cookbook.html
connectAPIKey <- Sys.getenv("RSC_API_KEY")

# Request a page of up to 25 usage records.
resp <- httr::GET(
  file.path(connectServer, "__api__/v1/instrumentation/shiny/usage?limit=100"),
  httr::add_headers(Authorization = paste("Key", connectAPIKey))

dantonnoriega

library(tidyverse)
dat <- as_tibble(mtcars) %>%
  mutate(vs = as.character(vs),
         am = as.character(am)) #just to make some non-numeric
dd0 <- dat %>%
  select(where(is_numeric)) %>%
  filter(if_any(disp:wt, ~ .x > 100))
dd0

library(data.table)

dantonnoriega

carryforward <- function(x, blank = is.na) {
  # SOURCE: https://stackoverflow.com/a/32536507/3987905
  # Find the values
  if (is.function(blank)) {
    isnotblank <- !blank(x)
  } else {
    isnotblank <- x != blank
  }
  # Fill down
  x[which(isnotblank)][cumsum(isnotblank)]

dantonnoriega

# github + git lfs api contenct extraction 
# GOAL: pull a large file that was sent to git lfs (cannot download directly from github.com)

## git-lfs API: https://github.com/git-lfs/git-lfs/tree/main/docs/api
## github API: https://docs.github.com/en/rest/reference/repos#get-repository-content

OWNER=dantonnoriega
REPO=some-repo
BRANCH=develop
# -----------------