This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
""" | |
DyNet implementation of a sequence labeler (POS taggger). | |
This is a translation of this tagger in PyTorch: https://gist.github.com/hal3/8c170c4400576eb8d0a8bd94ab231232 | |
Basic architecture: | |
- take words | |
- run though bidirectional GRU | |
- predict labels one word at a time (left to right), using a recurrent neural network "decoder" | |
The decoder updates hidden state based on: | |
- most recent word |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
install_submodule_git <- function(x, ...) { | |
install_dir <- tempfile() | |
system(paste("git clone --recursive", shQuote(x), shQuote(install_dir))) | |
devtools::install(install_dir, ...) | |
} | |
install_submodule_git("https://github.com/jonkeane/mocapGrip") |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# java -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=12605 Main # Name of .class program | |
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/pr/cloudera/parcels/CDH-5.5.2-1.cdh5.5.2.p0.4/lib/hadoop/lib/native | |
java -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=12611 -cp ta rget/da_record_linkage-0.0.1-SNAPSHOT-jar-with-dependencies.jar da_record_linkage.TestSnappy | |
netstat -plten | grep LISTEN | grep :120* # See if there is any open port |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
library(caret) | |
## select training indices preserving class distribution | |
in.train <- createDataPartition(yclass, p=0.8, list=FALSE) | |
summary(factor(yclass)) | |
ytra <- yclass[in.train]; summary(factor(ytra)) | |
ytst <- yclass[-in.train]; summary(factor(ytst)) | |
## standardize features: training parameters of scaling for test-part | |
Xtra <- scale(X[in.train,]) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Overwrite non-partitioned table with their own contents | |
CREATE table xx_COPY LIKE xx; | |
INSERT OVERWRITE TABLE xx | |
SELECT * FROM xx | |
# Overwrite partitioned table with their own contents | |
CREATE table xx_COPY LIKE xx; | |
SHOW PARTITIONS ABC; |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Reference: http://hadoop.apache.org/docs/r2.7.0/hadoop-project-dist/hadoop-common/FileSystemShell.html | |
# Show disk usage in human format | |
hadoop fs -du -s -h /user/hive/warehouse/da_cdepablo* | |
# Show permissions | |
hadoop fs -getfacl /user/hive/warehouse/da_cdepablo* | |
# Change permissions | |
hadoop fs -setfacl -R -m other::rwx /user/hive/warehouse/da_cdepablo |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#Number of active users per service - with a cutoff | |
SELECT `service`, COUNT(*) num_users | |
FROM | |
( | |
SELECT `service`, `hash_key`, COUNT(*) num_requests | |
FROM `log` | |
WHERE `date_operation` > '2014-12-01' | |
GROUP BY `service`, `hash_key` | |
ORDER BY num_requests DESC |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
// Use Gists to store code you would like to remember later on | |
console.log(window); // log the "window" object to the console |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/usr/bin/python | |
# -*- coding: utf-8 -*- | |
from lxml import html,etree | |
import requests | |
import unicodecsv | |
def group(iterator, count): | |
itr = iter(iterator) | |
while True: |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Credit http://stackoverflow.com/a/2514279 | |
for branch in `git branch -r | grep -v HEAD`;do echo -e `git show --format="%ci %cr" $branch | head -n 1` \\t$branch; done | sort -r |
NewerOlder