Skip to content

Instantly share code, notes, and snippets.

@ceteri
ceteri / Main.java
Last active Oct 6, 2015
Cascading for the Impatient, Part 5
View Main.java
public class
Main
{
public static void
main( String[] args )
{
String docPath = args[ 0 ];
String wcPath = args[ 1 ];
String stopPath = args[ 2 ];
String tfidfPath = args[ 3 ];
@ceteri
ceteri / Main.java
Created Jul 3, 2012
Cascading for the Impatient, Part 6
View Main.java
public class
Main
{
public static void
main( String[] args )
{
String docPath = args[ 0 ];
String wcPath = args[ 1 ];
String stopPath = args[ 2 ];
String tfidfPath = args[ 3 ];
@ceteri
ceteri / HTML
Created Jul 25, 2012
Cascading Sample Recommender
View HTML
<h1>Cascading Sample Recommender</h1>
<p>The goal for this project is to create a sample application in <a href="http://www.cascading.org/">Cascading 2.0</a> which shows how to build a simple kind of <a href="http://en.wikipedia.org/wiki/Recommender_system">social recommender</a>.</p>
<h2>Build</h2>
<p>First, clone a copy of the source code from our GitHub repo at <a href="https://github.com/Cascading/SampleRecommender">https://github.com/Cascading/SampleRecommender</a></p>
<pre><code>git clone https://github.com/Cascading/SampleRecommender.git
</code></pre>
@ceteri
ceteri / log
Created Sep 6, 2012
CMU Workshop on Cascading plus City of Palo Alto Open Data
View log
bash-3.2$ java -version
java version "1.6.0_33"
Java(TM) SE Runtime Environment (build 1.6.0_33-b03-424-11M3720)
Java HotSpot(TM) 64-Bit Server VM (build 20.8-b03-424, mixed mode)
bash-3.2$ hadoop -version
Warning: $HADOOP_HOME is deprecated.
java version "1.6.0_33"
Java(TM) SE Runtime Environment (build 1.6.0_33-b03-424-11M3720)
Java HotSpot(TM) 64-Bit Server VM (build 20.8-b03-424, mixed mode)
@ceteri
ceteri / log
Created Sep 7, 2012
Cascading wordcount sample
View log
bash-3.2$ cd cascading.samples/
bash-3.2$ ls
build.gradle hadoop logparser settings.gradle
build.xml loganalysis sample.build.gradle wordcount
bash-3.2$ cd wordcount/
bash-3.2$ ls
README.TXT build.gradle data src
bash-3.2$ java -version
java version "1.6.0_33"
Java(TM) SE Runtime Environment (build 1.6.0_33-b03-424-11M3720)
@ceteri
ceteri / pig.log
Created Sep 24, 2012
Debugging the Cascasding / Pig comparison - part 4
View pig.log
bash-3.2$ java -version
java version "1.6.0_35"
Java(TM) SE Runtime Environment (build 1.6.0_35-b10-428-11M3811)
Java HotSpot(TM) 64-Bit Server VM (build 20.10-b01-428, mixed mode)
bash-3.2$ pig -version
Warning: $HADOOP_HOME is deprecated.
Apache Pig version 0.10.0 (r1328203)
compiled Apr 19 2012, 22:54:12
bash-3.2$ cat src/scripts/wc.pig
@ceteri
ceteri / Impatient #1
Created Sep 25, 2012
Cascading user list questions
View Impatient #1
bash-3.2$ gradle clean jar
:clean
:compileJava
:processResources UP-TO-DATE
:classes
:jar
BUILD SUCCESSFUL
Total time: 4.316 secs
@ceteri
ceteri / log
Created Oct 14, 2012
ACM DM - Python exercise
View log
# use git to load ceteri-mapred (simplest as a ZIP)
# https://github.com/ceteri/ceteri-mapred
# cd to your ceteri-mapred download
Pacos-MacBook-Pro:ceteri-mapred ceteri$ ls
README doc graph.gephi src thresh.R
bin graph.csv msgs.tsv stopwords thresh.tsv
Pacos-MacBook-Pro:ceteri-mapred ceteri$ ls src/
map_filter.py map_parse.py map_wc.py red_filter.py red_idf.py red_wc.py util_extract.py util_gephi.py util_walk.py
Pacos-MacBook-Pro:ceteri-mapred ceteri$ head README
@ceteri
ceteri / log
Created Oct 14, 2012
ACM DM - Multitool exercise
View log
# use git to load multitool (simplest as a ZIP)
# https://github.com/Cascading/cascading.multitool
# to save time, we'll skip the JAR compile/build...
# download the JAR file from:
# https://s3.amazonaws.com/ceteri-mapred/multitool.jar
# cd to your cascading.multitool download
bash-3.2$ rm -rf output
bash-3.2$ hadoop jar ./multitool.jar source=data/days.txt select=Tuesday sink=output/tuesday.txt
@ceteri
ceteri / wc.clj
Created Oct 16, 2012
Cascading for the Impatient, Part 2 - Word Count
View wc.clj
; Paul Lam
; https://github.com/Quantisan/Impatient
(ns impatient.core
(:use [cascalog.api]
[cascalog.more-taps :only (hfs-delimited)])
(:require [clojure.string :as s]
[cascalog.ops :as c])
(:gen-class))