beppu

https://www.youtube.com/watch?v=fZGlmVMdKDs

Building Realtime Analytics Systems

Finding a solution

(Initially you might think…) Load all your data into Hadoop. Query it. Done

MapReduce/Hadoop has problems. It can be slow.

Not optimized for query latency

Need to introduce a query layer to make queries faster.

Making Queries Faster

What types of queries to optimize for?

beppu

<br />
<b>Notice</b>:  Undefined index: session_userid in <b>/Users/beppu/src/github.com/asg/top.php</b> on line <b>29</b><br />
<br />
<b>Notice</b>:  Undefined index: id in <b>/Users/beppu/src/github.com/asg/top.php</b> on line <b>38</b><br />
<br />
<b>Notice</b>:  Undefined index: current_document in <b>/Users/beppu/src/github.com/asg/top.php</b> on line <b>39</b><br />
<br />
<b>Notice</b>:  Undefined index: javascript_doc_id in <b>/Users/beppu/src/github.com/asg/top.php</b> on line <b>65</b><br />
<br />
<b>Notice</b>:  Undefined index: session_accessLevel in <b>/Users/beppu/src/github.com/asg/top.php</b> on line <b>70</b><br />

beppu

#!/usr/bin/env perl
use strict;
use warnings;
use LWP::Simple;
use URI;
use URI::QueryParam;
use URI::Escape;

while(<>) {
  my ($url, $title) = split(/\|/);

beppu

#!/usr/bin/env perl

use common::sense;
use AnyEvent;
use Coro;
use AnyEvent::HTTP::LWP::UserAgent;
use SOAP::Lite; # +trace => 'all';
use SOAP::Transport::HTTP;  # preload it so that we can set USERAGENT_CLASS reliably;

my @results;

beppu

[node@metanotes ~]$ perl -M5.10.0 -e 'say join("\n", split(":", $ENV{PATH}))' | xargs ls > path.txt
/home/node/local/bin:
npm
npm@0.2.12

/home/node/local/nodejs/bin:
node
node_g
node-repl
node-waf

beppu

mysql> select * from purchase_item where purchase_id = 2;
+----+-------------+-----+---------+------------+-------------+-------+---------------------+---------------------+
| id | purchase_id | day | t       | child_name | child_grade | price | created_on          | modified_on         |
+----+-------------+-----+---------+------------+-------------+-------+---------------------+---------------------+
| 37 |           2 |   0 | regular | Bill       |           0 |  5.00 | 2011-01-13 13:08:56 | 2011-01-13 15:08:56 | 
| 38 |           2 |   0 | regular | Bill       |           0 |  5.00 | 2011-01-13 13:08:56 | 2011-01-13 15:08:56 | 
| 39 |           2 |   0 | regular | Bill       |           0 |  5.00 | 2011-01-13 13:08:56 | 2011-01-13 15:08:56 | 
| 40 |           2 |   0 | regular | Bill       |           0 |  5.00 | 2011-01-13 13:08:56 | 2011-01-13 15:08:56 | 
| 41 |           2 |   0 | regular | Bill       |           0 |  5.00 | 2011-01-13 13:08:56 | 2011-01-13 15:08:56 | 
| 42 |           2 |   0 | regula

beppu

http://www.enterprisecorruption.com/

actor02-aug-01-12-11ss

  .. we are there to secure those nations on our citizen’s dimes..

for the financial benefit of our citizen’s so-called.. economic “competitors” in china

 

beppu

#!/usr/bin/perl
use common::sense;
use Web::Scraper;
use Data::Dump 'pp';
use URI;
use IO::All;
use aliased 'Squatting::H';

my $url_pattern = "http://www.enterprisecorruption.com/?page_id=%d";
my @id          = qw( 41 42 11 43 44 50 51 45 9 46 47 48 10 49 );

beppu

/**
 *
 */
package com.nugporn.xxx;

import java.io.IOException;
import java.util.ArrayList;

import javax.xml.parsers.DocumentBuilder;
import javax.xml.parsers.DocumentBuilderFactory;

beppu

#!/usr/bin/env perl
use common::sense;
use EV;
use AnyEvent;
use AnyEvent::DNS;
use Coro;
use Coro::AnyEvent;
use Data::Dump 'pp';
use Getopt::Long;