Skip to content

Instantly share code, notes, and snippets.

View mcfunley's full-sized avatar

Dan McKinley mcfunley

View GitHub Profile
require 'helpers/analytics'
analytics_cascade do
analytics_flow do
analytics_source 'visit_logs'
assembly 'visit_logs' do
unsequence 'visit', :output => ['event']
query_event 'event', 'visit_id', 'event_type', 'query', 'epoch_ms'
dbadmin=> select count(*) from weblog.events;
count
-------
0
(1 row)
[/Users/dmckinley/development/tmp]
dmckinley@ETSYNY-L479 $ cat test.scala
object Test {
def main(args : Array[String]) {
for (x <- (1 until 20)) {
println(x)
}
}
}
diff -ur twitter-scalding/README.md etsy-scalding/README.md
--- twitter-scalding/README.md 2013-05-29 23:08:00.000000000 -0700
+++ etsy-scalding/README.md 2013-05-29 23:08:00.000000000 -0700
@@ -4,7 +4,7 @@
Scalding is built on top of [Cascading](http://www.cascading.org/), a Java library that abstracts away much of the complexity of Hadoop.
-Current version: 0.8.0
+Current version: 0.8.1

<gfodorbot> my point is you apparently can't read sequence files without the native libraries for decompression <dan> what is amazing about that is fodor just said something pertinent to what i'm working on RIGHT NOW

### Before using:
### pear install doc.php.net/pman
(defun php-pman ()
(interactive)
(let ((man-term (read-from-minibuffer "Term: " (current-word))))
(man (concat "-M /usr/share/pear/doc/pman " man-term))))
(add-hook 'php-mode-hook '(lambda () (local-set-key "\C-c\C-h" 'php-pman)))
dmckinley@dmckinley $ hadoop fs -ls /logs.etsy.com/event_logs/2013_01_30 | awk '{ sum += $5 } END { print sum / 1024 / 1024 }'
74042.1
14 97954900
16 96457161
32 95590098
28 93552850
9 89226804
34 86835728
8 84536021
33 80861650
22 79238941
18 78353033
// Purchases: these cause a proliferation of events because there
// are a minimum of three entities in the system that people can
// be interested in that take part: the buyer, the shop, and the
// item.
//
// For consistency we'll generate all of these activities, but it
// wouldn't make sense to display all of them. People interested
// in an item would see (item, purchased_by, buyer). People
// interested in a shop would see (shop, sold_to, buyer), maybe
// aggregated to show several sales. People interested in a
(eval-when-compile (require 'cl))
(require 'w3m-load)
(require 'w3m)
(require 'dom)
(require 'php-mode)
(setq w3m-use-cookies t)
(defvar *url-handlers*