Skip to content

Instantly share code, notes, and snippets.

View soodoku's full-sized avatar

gaurav soodoku

254 followers · 10 following
View GitHub Profile
@soodoku
soodoku / mturk.ipynb
Created August 31, 2018 17:40
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@soodoku
soodoku / read_parsed_dmoz.py
Created February 18, 2021 20:25
Reading in the parsed DMOZ file
import csv
import pandas as pd
import numpy as np
df = pd.read_csv('parsed-new.csv', header = None, delimiter="\t", quoting=csv.QUOTE_NONE, encoding='utf-8')
df.head()
@soodoku
soodoku / county_dma_2016.R
Created November 21, 2017 19:11
DMA to County for 2016
library(readr)
library(dplyr)
a_string <- read_file("nielsen_2016")
split_lines <- strsplit(a_string, "\r\n")[[1]]
split_cols <- strsplit(split_lines, "--")
dat_frame <- ldply(split_cols)
names(dat_frame) <- c("dma", "counties")
write.csv(dat_frame, file = "dma_counties_2016.csv", row.names = F)
@soodoku
soodoku / forest_lasso.ipynb
Last active July 4, 2023 05:45
Post Process RF Using Lasso
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@soodoku
soodoku / group_interest.R
Last active July 9, 2023 02:09
Group Interest Partisans?
# Pareto Party
# Load libs
library(tidyverse)
library(readstata13)
library(car)
library(dplyr)
library(xtable)
library(dplyr)
@soodoku
soodoku / not_normal.ipynb
Last active July 16, 2023 04:58
Not Normal
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@soodoku
soodoku / flask_app.py
Last active July 16, 2023 17:36
Remote Logging of Errors With User Approval
from flask import Flask, request
import uuid
app = Flask(__name__)
@app.route('/error_endpoint', methods=['POST'])
def receive_error_message():
error_message = request.form.get('error_message')
if error_message:
@soodoku
soodoku / get_unique_domain_names_from_comscore.py
Created February 12, 2018 02:15
Get a list of unique domain names from comScore browsing data
#
# Get All Unique Domain Names from comScore
#
# INPUT: comScore browsing data file
#
# OUTPUT: a text file containing a list of unique domains
#
# PAREMETERS:
# + INTERNET_USAGE_FILE: path to the comScore browsing data
# + FINAL_OUTPUT_FILE: path to intended output file
@soodoku
soodoku / zillowScraper.js
Created September 14, 2023 20:47 — forked from adrianhorning08/zillowScraper.js
Zillow Scraper
async function scrollDown() {
const wrapper = document.querySelector("#search-page-list-container");
await new Promise((resolve, reject) => {
var totalHeight = 0;
var distance = 600;
var timer = setInterval(async () => {
var scrollHeightBefore = wrapper.scrollHeight;
wrapper.scrollBy(0, distance);
totalHeight += distance;
@soodoku
soodoku / Cable_penetration_1.csv
Last active September 25, 2023 21:52
Cable Ownership by DMA by Race Over the Years
We can make this file beautiful and searchable if this error is corrected: It looks like row 6 should actually have 34 columns, instead of 1. in line 5.
dma_name,dma_rank,dma_crossover,state,dma_code_mp90_or_shpfile,from_mp90_or_shpfile,2010,2009,2008,2007,2006,2005,2003,2002,2001,2000,1999,1998,1997,1996,1995,1994,1993,1992,1991,1990,1989,1988,1987,1986,1985,1984,1983,1974
Abilene-Sweetwater,163,ABILENE (TX) - SWEETWATER (TX) ,TX,662,ABILENE-SWEETWATER,42,42,47,57,58,61,63,66,67,71,71,71,70,69.2,68.2,67.6,69.3,68,69.3,67,67.2,65.2,69.5,67.8,67.8,66.5,61.3,44
Ada-Ardmore,,,,657,ADA-ARDMORE,,,,,,,,,,,,,,,,53.5,51.1,51.3,52.4,50.7,52,48.4,50,51.8,51.8,51,50.1,24
"Albany, GA",148,ALBANY (GA) ,GA,525,"ALBANY, GA",59,61,60,66,67,66,68,70,62,62,62,63,61,60.2,59.2,59.2,59.1,58.4,61,58,55.3,52.2,53.3,48.3,48.3,49.9,45.8,16
Albany-Schenectady-Troy,55,ALBANY (NY) - SCHENECTADY (NY) - TROY (NY) ,NY,532,ALBANY-SCHENECTADY-TROY,76,77,78,79,79,79,79,78,75,75,74,74,73,71.3,67.4,68.3,67.6,67.9,68.5,66,64.2,60.8,58.8,56.2,56.2,53.2,49.2,8
Albuquerque-Santa