mumrah

public class SolrUpdater implements Runnable {
  private final SolrInputDocument doc = new SolrInputDocument();
  private final UpdateRequest req = new UpdateRequest();
  private final SolrServer solr;
  private final BlockingQueue<String> strings;
  private final AtomicLong id;

  public SolrUpdater(SolrServer solr, BlockingQueue<String> strings,
      AtomicLong id) {
    this.solr = solr;

mumrah

      a !? (2000, WantNext("group", "test")) match {
        case m1: Some[NextMessage] => 
          println("case Some[NextMessage]")
          println(m1)
        case m1: Some[NoMessage] => 
          println("case Some[NoMessage]")
          println(m1)
        case m1: Option[Nothing] =>
          println("case Option[Nothin]")
          println(m1)

mumrah

class PongActor extends Actor {
    def act() {
      val y = 4
      loop {
        react { 
          case Ping(2) =>
            println("got ping(2)")
            sender ! Pong
          case Ping(y) =>
            println("got ping(y)")

mumrah

REST interface for Kafka

Taking inspiration from the projects page...

I think it would really useful and pretty simple to add a REST interface to Kafka. I could see two possible routes (not mutually exclusive): using HTTP as a dumb transport layer, and using it with different media types for application-friendly consumption of messages (JSON, XML, etc). The dumb transport would be useful for languages without first-class clients, and the content-type extension would be useful for writing web apps that are Kafka-enabled.

HTTP as a transport

Consuming data

mumrah

$ /opt/mahout-distribution-0.6/bin/mahout kmeans -h
MAHOUT_LOCAL is not set; adding HADOOP_CONF_DIR to classpath.
no HADOOP_HOME set, running locally
Class: org.apache.mahout.driver.MahoutDriver
Args: kmeans -h
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/mahout-distribution-0.6/mahout-examples-0.6-job.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/mahout-distribution-0.6/lib/slf4j-jcl-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/mahout-distribution-0.6/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.

mumrah

echo "IyEvYmluL2Jhc2gKKGJhc2U2NCAtZCB8IHB5dGhvbikgPDxFT0YKY0hKcGJuUW9JbXRwYkd3Z2JXVWlLUW89CkVPRgo=" | base64 -d | bash

mumrah

#!/bin/bash

(base64 -d | bash) <<EOF
IyEvYmluL2Jhc2gKCmVjaG8gImhlbGxvLCB3b3JsZCIK
EOF

mumrah

    File propFile = new File("oozie.action.output.properties");
    Properties props = new Properties();
    props.put("foo", "bar");
    OutputStream os = new FileOutputStream(propFile);
    props.store(os, "");
    os.close();
    

mumrah

BEGIN {
    RS = ORS = "\r\n"
    HttpService = "/inet/tcp/8000/0/0"
    while(1) {
	_NR = 0
	while((HttpService |& getline) > 0) {
	    _NR += 1
	    handle_header($0)
	}
	close(HttpService)

mumrah

BEGIN {
    FS=",";
    OFS=",";
}

{
    COUNTS[$1] += 1;
    TOTALS[$1] += $2;
}