Skip to content

Instantly share code, notes, and snippets.

View nassimhaddad's full-sized avatar

Nassim Haddad nassimhaddad

View GitHub Profile
@nassimhaddad
nassimhaddad / encoding.md
Created October 20, 2014 09:55
how to determine the encoding of a csv file ?

Source: http://pandaproject.net/docs/determining-the-encoding-of-a-csv-file.html

If you have no way of finding out the correct encoding of the file, then try the following encodings, in this order:

  • utf-8
  • iso-8859-1 (also known as latin-1) (This is the encoding of all census data and much other data produced by government entities.)
  • utf-16

If none of these work the likelihood you are going to determine the encoding without additional information from the source is very low. In theory you may be able to guess the encoding based on the language of the author, however this not a recommended practice.

@nassimhaddad
nassimhaddad / yearweek.R
Created October 15, 2014 11:40
frequently used translation of a date to a yearweek
yearweek <- format(as.Date("2010-09-01"), format = "%G%V")
@nassimhaddad
nassimhaddad / README.md
Last active August 29, 2015 14:07
running shiny apps as server

launch the shiny server in the background from the linux console (Rscript launch_app.R &),
and then when it says: listening on http://127.0.0.1:8787, you can access it via:

http://IP:8787/p/7815/

(where 7815 is the port specified in the .R file)

@nassimhaddad
nassimhaddad / vec.translate.R
Created October 3, 2014 13:27
Small tools to quickly and easily translate data values using dictionaries.
#'
read.markdown.table <- function(string){
require(stringr)
temp <- (strsplit(string, "\n")[[1]])
temp <- temp[sapply(temp,nchar)>0]
temp <- strsplit(temp, "\\|")
temp <- lapply(temp, function(x)str_trim(x[-1]))
title_row <- temp[[1]]
temp[[1]]<-NULL
@nassimhaddad
nassimhaddad / distort.R
Created September 4, 2014 10:12
to create a distortion in a time series
#' function that distorts
distort <- function(y, from, to){
before <- seq(from=1, to=from,length.out = to)
after <- seq(from=from+1, to=length(y), length.out = length(y)-to)
out <- approx(seq(y),y, xout = c(before, after))
return(out$y)
}
# example
x <- seq(1,120, by = 1)
@nassimhaddad
nassimhaddad / ggvis_densities.R
Created July 15, 2014 06:19
multiple densities in a chart with ggvis
library(ggvis)
mtcars %>% ggvis(~mpg, fill=~as.factor(cyl)) %>% group_by(cyl) %>% layer_densities()
@nassimhaddad
nassimhaddad / package_dependencies.R
Last active August 29, 2015 14:03
download dependent packages. useful to create a production environment, manage versions, and working on machines with no internet connections.
# Install https://github.com/andrie/miniCRAN
require(tools)
require(miniCRAN)
repositorylocation <- "/home/Downloads"
pks <- package_dependencies(c("data.table","ffbase"), db = available.packages(),
which = c("Depends","Imports","LinkingTo"), recursive=TRUE)
pks <- unique(c(do.call(c, pks), c("data.table","ffbase")))
makeRepo(pkgs=pks, path=repositorylocation, type = "source", download = TRUE, writePACKAGES = TRUE)
install.packages(pkgs = "data.table", repos = repositorylocation, type = "source",
@nassimhaddad
nassimhaddad / perfCompare.R
Created June 29, 2014 07:49
compare performance of models
# code copied from:
# http://heuristically.wordpress.com/2009/12/23/compare-performance-machine-learning-classifiers-r/
# load the ROCR package which draws the ROC curves
require(ROCR)
# create an ROCR prediction object from rpart() probabilities
x.rp.prob.rocr <- prediction(x.rp.prob[,2], BreastCancer[ind == 2,'Class'])
# prepare an ROCR performance object for ROC curve (tpr=true positive rate, fpr=false positive rate)
x.rp.perf <- performance(x.rp.prob.rocr, "tpr","fpr")
@nassimhaddad
nassimhaddad / interactiveScatter.R
Last active August 29, 2015 14:02
interactive scatterplot in r with tooltips
interactiveScatter <- function(xvar, yvar, data,
tooltip = c(xvar, yvar), title = "",
lmline = TRUE){
require(rCharts)
x <- xvar
y <- yvar
tooltipString <- paste0("#!function(item){return ",
@nassimhaddad
nassimhaddad / addConsol.py
Created June 17, 2014 09:54
launch ipython console alongside notebook
%qtconsole