Skip to content

Instantly share code, notes, and snippets.

@ctufts
ctufts / java_compatable_regex.txt
Last active September 8, 2016 20:15
Regular expression examples in R
(?<!@|#)\b\w+ : Remove all words starting with @ or # (remove hashtags and user handles from twitter)
(?<!@|#)\b\w{2,} : Same as above but only keep words with length of 2 or greater
@ctufts
ctufts / PhlCrime_GettingStarted_PT_I_II.R
Last active November 7, 2015 22:35
The script imports Philadelphia Crime Data Parts I and II (https://www.opendataphilly.org/dataset/crime-incidents), creates a summary based on year, month, and crime type and creates a basic map in leaflet using the first 1000 incidents
# the following script will import Philadelphia Crime Data
# Parts I and II, create a summary based on year, month, and crime type
# and will create a basic map in leaflet using the first 1000 incidents
# you will need to install dplyr, leaflet, readr, lubridate, and stringr
# packages ( install.packages('package name') )
rm(list = ls())
library(dplyr)
library(leaflet)
library(readr)
@ctufts
ctufts / twitterStream_to_mysql.py
Last active July 8, 2019 01:37
The script creates a twitter stream listener using Tweepy then stores the data in a MySQL database. The code works with Python 3 and will also support emoji ( utf8mbr4 encoding ).
# code for streaming twitter to a mysql db
# for Python 3 and will support emoji characters (utf8mb4)
# based on the Python 2 code
# supplied by http://pythonprogramming.net/twitter-api-streaming-tweets-python-tutorial/
# for further information on how to use python 3, twitter's api, and
# mysql together visit: http://miningthedetails.com/blog/python/TwitterStreamsPythonMySQL/
from tweepy import Stream
from tweepy import OAuthHandler
@ctufts
ctufts / replaceValuesWithKeys.py
Last active November 7, 2015 22:39
Find 'value' in a list of tokens and replace them with the 'key' of a dictionary. The dictionary can have multiple values per key. Example use: replacing multiple nick names with one common name.
s = ['c' ,'is', 'equal', 'to', 'b']
print(s)
# output >> ['c', 'is', 'equal', 'to', 'b']
# dictionary of names:values
d = {'joe':['a', 'b'], 'tom':['c', 'd']}
# replace any values from the dict with the key value
for i in range(0, len(s)):
for key,value in d.items():
for v in value:
##################### Import Libraries ##############################################
# if you don't have the libraries below you can install the library using the
# following command : install.packages('package_name_here_in_quotes')
library(dplyr)
library(lubridate)
library(RColorBrewer)
library(leaflet)
library(stringr)
library(rgdal)
@ctufts
ctufts / deepAssignmentOperatorExamples.R
Created April 28, 2016 19:56
Example illustrating the deep assignment operator's (<<-) use in R
####################################################
# case of global assignment
# parent environment is the global environment
# from Hadley Wickham's 'Advanced R'
#####################################################
x <- 0
f <- function() {
x <<- 1
}
@ctufts
ctufts / LinuxCommands.md
Last active March 1, 2023 20:09
Common linux commands

Ubuntu Linux

  • sudo -i : elevate to super user
  • du : get breakdown of memory usage of all subdirectories
  • df -h : get breakdown of memory usage on disk
  • ls -a : show all files in directory (including hidden files)
  • rsync : copy files from one server to another (similar to scp but more functionality)
    • Set up rsync with sudo
    • rsync -az -e "ssh" --rsync-path="sudo rsync" user@servername:/pulled-source-directory /local-directory/
    • rsync [source] [destination]
@ctufts
ctufts / General Conda Commands.md
Last active January 3, 2024 03:36
List of commonly used commands in anaconda
  • conda info --envs : lists all environments
  • source activate <env name>: activate an environment
  • source deactivate: deactivate an environment
  • conda list : list all packages installed
  • conda create --name <env name> python=3 astroid babel : create new environment, specify version of python, and install packages
  • WINDOWS NOTE: SOURCE is not recognized. When deactivating and activating in the anaconda command prompt, skip source and just type deactivate or activate depending on what you are trying to do.
  • conda env export > environment.yml: export conda environment requirements list to a file
  • conda env remove -n ENV_NAME : delete environment
@ctufts
ctufts / ODS.md
Last active August 15, 2016 16:03
Open data sites
@ctufts
ctufts / .block
Last active April 15, 2019 23:12
D3 Scatterplot with Regression Line
license: gpl-3.0
height: 500
scrolling: no
border: no