petro-rudenko

petro-rudenko

(def nlp
  (let [props (new java.util.Properties)]
    (.setProperty props "annotators" "tokenize,ssplit,parse,sentiment")
    (new edu.stanford.nlp.pipeline.StanfordCoreNLP props)))

(defn find-sentiment
  "Determines the sentiment of each sentence in a given glob of text. Results in a collection of integers ranging from [0-4]: where 0 is 'Very negative', 2 is 'neutral', and 4 is 'Very positive'"
  [blob]
  (let [glob (apply str blob)]
    (let [main-sentiment 0

petro-rudenko

//Interface we need to implement

public interface DataworkerView {
	public User getUser();

	
	//E.g. String fs = getConfigValue("dataworker.defaultFs")
	// fs -> "webhdfs://example.com:14000/"
	public String getConfigValue(String configName);
		

petro-rudenko

1. General Background and Overview

• Probabilistic Data Structures for Web Analytics and Data Mining : A great overview of the space of probabilistic data structures and how they are used in approximation algorithm implementation.
• Models and Issues in Data Stream Systems
• Approximate Frequency Counts over Data Streams by Gurmeet Singh Manku & Rajeev Motwani : One of the early papers on the subject.
• Methods for Finding Frequent Items in Data Streams by Graham Cormode & Marios Hadjieleftheriou
• The space complexity of approximating the frequency moments by Noga Alon, Yossi Matias, Mario Szegedy : one of the most influential papers introducing succinctn

petro-rudenko

• http://christophermeiklejohn.com/distributed/systems/2013/07/12/readings-in-distributed-systems.html
• http://michaelrbernste.in/2013/11/06/distributed-systems-archaeology-works-cited.html
• http://dancres.org/reading_list.html
• http://rxin.github.io/db-readings/
• http://research.microsoft.com/en-us/um/people/lamport/pubs/pubs.html
• http://www.eecs.berkeley.edu/GradAffairs/CS/Prelims/db.html
• http://pdos.csail.mit.edu/dsrg/papers/
• http://scalingsystems.com/2011/09/07/reading-list-for-distributed-systems/
• http://courses.engr.illinois.edu/cs525/sp2011/sched.htm
• http://henryr.github.io/distributed-systems-readings/

petro-rudenko

(use 'clojure.core.async)


;this is the function you want to use
(defn lazy-channels [chs]
   (lazy-seq (cons (let [ [v _] (alts!! chs)] v) (lazy-channels chs))))


;now for the tesging
(def chs [ (chan) (chan) (chan) ]) ; the channels can come from anywhere, here we are using three channels for testing

petro-rudenko

;;
;; NS CHEATSHEET
;;
;; * :require makes functions available with a namespace prefix.
;;
;; * :use makes functions available without a namespace prefix
;;   (i.e., refers functions to the current namespace).
;;
;; * :import refers Java classes to the current namespace.
;;

petro-rudenko

(defn buffered-select [f-select init-pos]
  "Creates a lazy sequence of messages for this datasource"
  (letfn [  
           (m-seq [buff pos] 
                   (let [buff2 (if (empty? buff) (f-select pos) buff)]
                         (cons (first buff2) (lazy-seq (m-seq (rest buff2) (inc pos) )))         
                     )
                   )
           ]

petro-rudenko

(ns proc
  (:import [java.lang ProcessBuilder])
  (:use [clojure.java.io :only [reader writer]]))

(defn spawn [& args]
  (let [process (-> (ProcessBuilder. args)
                  (.start))]
    {:out (-> process
            (.getInputStream)
            (reader))

petro-rudenko

Tutorials

• The best tutorials are in the introductory books. See below.
• Getting Started with Clojure - A detailed tutorial on getting a modern (as of Jan 2013) Clojure workflow going.
  • Emacs Live is a nice development environment based on Emacs.
• Understanding The Clojure Development Ecosystem
• Clojure Docs Site is a community-driven doc site with good tutorials, and reference material going somewhat deeper than individual API docs.
• Functional Programming for the Rest of Us is a classic introduction to functional thinking
• [A comprehensive article on namespaces and different ways of requiring them](http://blog.8thlight.com/colin-jones/2010/12/05/clojure-libs-and-namespaces-require-use-import-and-ns.