Skip to content

Instantly share code, notes, and snippets.

@rsumbaly
rsumbaly / KStreamImplicits.txt
Last active September 10, 2017 14:52
Making it easier to write kstream constructs in scala
import java.util.Comparator
import org.apache.kafka.streams.kstream._
import org.apache.kafka.streams.processor.StreamPartitioner
object KStreamImplicits {
implicit def scalaFunctionToPredicate[K, V](f: (K, V) => Boolean) = new Predicate[K, V] {
override def test(k: K, v: V): Boolean = f(k, v)
}
package kafka.schemaregistry.util
import junit.framework.Assert._
import org.junit.Test
import org.scalatest.junit.JUnit3Suite
import java.util.TreeSet
class ZNodeNameTest extends JUnit3Suite {
@rsumbaly
rsumbaly / gist:1260230
Created October 3, 2011 20:58
Getting YARN ( from MRv2 ) running locally
svn co http://svn.apache.org/repos/asf/hadoop/common/trunk hadoop-trunk -r 1178440
cd hadoop-trunk/
mvn clean install package -Dtar -Pdist -DskipTests -P-cbuild
cd hadoop-mapreduce-project/
mvn clean install package -Dtar -Pdist -DskipTests -P-cbuild assembly:assembly
mkdir ~/HADOOP_MAPREDUCE
cp ./../../hadoop-trunk/hadoop-common-project/hadoop-common/target/hadoop-common-0.24.0-SNAPSHOT.tar.gz ~/HADOOP_MAPREDUCE/.
cp ./../../hadoop-trunk/hadoop-mapreduce-project/target/hadoop-mapreduce-0.24.0-SNAPSHOT-all.tar.gz ~/HADOOP_MAPREDUCE/.
cd ~/HADOOP_MAPREDUCE
tar xvfz hadoop-common-0.24.0-SNAPSHOT.tar.gz ; mv hadoop-common-0.24.0-SNAPSHOT common
package voldemort.utils;
import java.io.File;
import java.io.IOException;
import java.text.DecimalFormat;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.Map.Entry;
package voldemort.utils;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.util.HashMap;
import java.util.Iterator;
import java.util.List;
import java.util.Set;
public static void partitionLoad(Cluster cluster, List<StoreDefinition> storeDefs) {
HashMap<StoreDefinition, Integer> uniqueStoreDefs = getUniqueStoreDefinitionsWithCounts(storeDefs);
for(Entry<StoreDefinition, Integer> storeDefCounts: uniqueStoreDefs.entrySet()) {
StoreDefinition storeDef = storeDefCounts.getKey();
RoutingStrategy strategy = new RoutingStrategyFactory().updateRoutingStrategy(storeDef,
cluster);
HashMap<Integer, Integer> partitionToCountMapping = Maps.newHashMap();
for(int partitionId = 0; partitionId < cluster.getNumberOfPartitions(); partitionId++) {
List<Integer> partitionList = strategy.getReplicatingPartitionList(partitionId);
@rsumbaly
rsumbaly / gist:985239
Created May 22, 2011 07:05
Yes, I've started writing JS now :|
$( function () {
globalData = [];
var placeholder = $("#placeholder");
$.plot(placeholder, [[]], options);
var num_points = 300;
function onDataReceived(data_point) {
globalData.push(data_point.data + ( Math.random() * 10));
var res = [];
@rsumbaly
rsumbaly / gist:969724
Created May 13, 2011 00:12
Doesn't work....Maybe does..
package voldemort;
import java.util.HashMap;
import java.util.LinkedList;
import java.util.List;
import voldemort.cluster.Cluster;
import voldemort.cluster.Node;
import voldemort.cluster.Zone;
import voldemort.routing.RoutingStrategy;
@rsumbaly
rsumbaly / gist:965695
Created May 11, 2011 00:41
FileStreamTest
package voldemort.performance;
import java.io.BufferedWriter;
import java.io.File;
import java.io.FileOutputStream;
import java.io.FileWriter;
import java.io.IOException;
import java.util.Arrays;
import java.util.Collection;
import java.util.HashMap;
package voldemort.client.rebalance;
import java.util.ArrayList;
import java.util.Collections;
import java.util.HashSet;
import java.util.Iterator;
import java.util.List;
import java.util.Map;
import java.util.Queue;
import java.util.Set;