Skip to content

Instantly share code, notes, and snippets.

@nipra
nipra / notes.txt
Created November 13, 2012 10:04
CDH4 Hadoop + HBase Pseudo-distributed Mode installation
# Installing CDH4 on a Single Linux Node in Pseudo-distributed Mode
# https://ccp.cloudera.com/display/CDH4DOC/Installing+CDH4+on+a+Single+Linux+Node+in+Pseudo-distributed+Mode
# Installing CDH4 with MRv1 on a Single Linux Node in Pseudo-distributed mode
# On Ubuntu and other Debian systems
nipra@lambda:Downloads$ wget -cv http://archive.cloudera.com/cdh4/one-click-install/precise/amd64/cdh4-repository_1.0_all.deb
nipra@lambda:Downloads$ sudo dpkg -i cdh4-repository_1.0_all.deb # Adds /etc/apt/sources.list.d/cloudera-cdh4.list ??
nipra@lambda:Downloads$ dpkg -L cdh4-repository # To view the files on Ubuntu systems
# Install CDH4
@terrancesnyder
terrancesnyder / kafka-consumer-example.java
Last active October 15, 2015 11:03
Example of processing Kafka messages using JQuery like deferred / promise for cleaner async code.
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.util.List;
import java.util.Map;
import java.util.Properties;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import org.apache.avro.io.BinaryDecoder;
@mushkevych
mushkevych / Dockerfile
Last active December 30, 2015 07:39
Docker CDH 4.5
FROM ubuntu:precise
MAINTAINER Bohdan Mushkevych
# Installing Oracle JDK
RUN apt-get -y install python-software-properties ;\
add-apt-repository ppa:webupd8team/java ;\
apt-get update && apt-get -y upgrade ;\
echo oracle-java7-installer shared/accepted-oracle-license-v1-1 select true | /usr/bin/debconf-set-selections ;\
apt-get -y install oracle-java7-installer && apt-get clean ;\
update-alternatives --display java ;\
@johnynek
johnynek / gist:8290375
Created January 6, 2014 21:47
example of LAG type function in the scalding Fields API (similar for typed)
groupBy('source) {
_.sortBy('links)
.reverse
.mapStream[(String,Int), (String, Int, Int, Int)]
(('destination, 'links) -> ('destination, 'links, 'rank, 'gap)) { destLinks =>
destLinks.scanLeft(None: Option[(String, Int, Int, Int)]) {
(prevRowOut: Option[(String,Int,Int,Int)], thisRow: (String, Int)) =>
val (dest, links) = thisRow
prevRowOut match {
case None => Some((dest, links, 1, 0)) // rank 1, gap 0 -- not exactly what you wanted...
@mrserverless
mrserverless / MultiTenantDataSource
Last active February 17, 2017 09:52
Dropwizard MultiTenant DataSource using RequestScope injected Schema object
public class MultiTenantDataSource implements ManagedDataSource {
private final ManagedDataSource managedDataSource;
private final SchemaResolver schemaResolver;
public MultiTenantDataSource( final ManagedDataSource managedDataSource, final SchemaResolver schemaResolver )
{
this.managedDataSource = managedDataSource;
this.schemaResolver = schemaResolver;
}
@terrancesnyder
terrancesnyder / kafka-consumer.java
Created February 8, 2014 16:12
Simple example of publishing avro messages to Kafka.
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.util.List;
import java.util.Map;
import java.util.Properties;
import org.apache.avro.io.BinaryDecoder;
import org.apache.avro.io.BinaryEncoder;
import org.apache.avro.io.DecoderFactory;
@johnfrey99
johnfrey99 / JacksonConverter
Created July 9, 2013 19:32
Retrofit converter based on Jackson instead of Gson.
public class JacksonConverter implements Converter
{
private final ObjectMapper mapper;
public JacksonConverter(ObjectMapper mapper)
{
this.mapper = mapper;
}
@Override public Object fromBody(TypedInput body, Type type) throws ConversionException
@visenger
visenger / install_scala_sbt.sh
Last active January 31, 2023 19:10
Scala and sbt installation on ubuntu 12.04
#!/bin/sh
# one way (older scala version will be installed)
# sudo apt-get install scala
#2nd way
sudo apt-get remove scala-library scala
wget http://www.scala-lang.org/files/archive/scala-2.11.4.deb
sudo dpkg -i scala-2.11.4.deb
sudo apt-get update
@GaryRogers
GaryRogers / RSyslogSetup.md
Last active October 5, 2023 22:51
RSyslog Forwarding Setup

RSyslog Forwarding Setup

Overview

I'm looking to centralize logging for our dev team into Elasticsearch via Logstash. The wrinkle is that we aren't a Java shop, so installing java on our hosts just to ship logs back to a central Logstash indexer is something we'd like to avoid. So, I'm approaching things as a chance to understand RSyslog and its capabilities as a log shipper.

Procedure

Set up TCP listening on the log index host

Uncomment the following lines in /etc/rsyslog.conf. This will enable the rsyslog daemon to listen for incoming requests on TCP port 514. We're using TCP here so that we can have some confidence that the messages from the agent hosts reach the indexer. (More on this below)

@mislav
mislav / pagination.md
Created October 12, 2010 17:20
"Pagination 101" by Faruk Ateş

Pagination 101

Article by Faruk Ateş, [originally on KuraFire.net][original] which is currently down

One of the most commonly overlooked and under-refined elements of a website is its pagination controls. In many cases, these are treated as an afterthought. I rarely come across a website that has decent pagination, and it always makes me wonder why so few manage to get it right. After all, I'd say that pagination is pretty easy to get right. Alas, that doesn't seem the case, so after encouragement from Chris Messina on Flickr I decided to write my Pagination 101, hopefully it'll give you some clues as to what makes good pagination.

Before going into analyzing good and bad pagination, I want to explain just what I consider to be pagination: Pagination is any kind of control system that lets the user browse through pages of search results, archives, or any other kind of continued content. Search results are the o