rajkrrsingh

$wget https://github.com/google/protobuf/releases/download/v2.5.0/protobuf-2.5.0.tar.bz2
$tar xvf protobuf-2.5.0.tar.bz2
$cd protobuf-2.5.0
$./configure CC=clang CXX=clang++ CXXFLAGS='-std=c++11 -stdlib=libc++ -O3 -g' LDFLAGS='-stdlib=libc++' LIBS="-lc++ -lc++abi"
$make -j 4 
$sudo make install
$protoc --version

zoltanctoth

from pyspark.sql.types import StringType
from pyspark.sql.functions import udf

maturity_udf = udf(lambda age: "adult" if age >=18 else "child", StringType())

df = spark.createDataFrame([{'name': 'Alice', 'age': 1}])
df.withColumn("maturity", maturity_udf(df.age))

df.show()

kwlzn

Building a test pex

setup.py

yakuza:src kw$ cat testapp/setup.py
#!/usr/bin/env python

from distutils.core import setup

chetan

#!/usr/bin/env bash

# list_hadoop_codecs.sh
#
# USAGE:
# curl -sL https://gist.github.com/chetan/6524829/raw/list_hadoop_codecs.sh | bash

# make sure hadoop is avail
if [[ -z `which hadoop 2>/dev/null` ]]; then
  echo "hadoop command not found!"

loicdescotte

Scala for comprehension translation helper

"For comprehension" is a another syntaxe to use map, flatMap and withFilter (or filter) methods.

yield keyword is used to aggregate values in the resulting structure.

This composition can be used on any type implementing this methods, like List, Option, Future...

UniIsland

#!/usr/bin/env python

"""Simple HTTP Server With Upload.

This module builds on BaseHTTPServer by implementing the standard GET
and HEAD requests in a fairly straightforward manner.

"""

jboner

Latency Comparison Numbers (~2012)
----------------------------------
L1 cache reference                           0.5 ns
Branch mispredict                            5   ns
L2 cache reference                           7   ns                      14x L1 cache
Mutex lock/unlock                           25   ns
Main memory reference                      100   ns                      20x L2 cache, 200x L1 cache
Compress 1K bytes with Zippy             3,000   ns        3 us
Send 1K bytes over 1 Gbps network       10,000   ns       10 us
Read 4K randomly from SSD*             150,000   ns      150 us          ~1GB/sec SSD

wickman

Python development using Pants

brian wickman - @wickman

[TOC]

Why use Pants for Python development?

Pants makes the manipulation and distribution of hermetically sealed Python environments

chadselph

from collections import defaultdict

class BadMatch(NameError):
    """Exception when your args don't match a pattern"""
    pass

class Any(object):
    """
    >>> 'wutup' == Any()
    True

bdarnell

#!/usr/bin/env python
"""This is a demonstration of sharing file descriptors across processes.

It uses Tornado (need a recent post-2.0 version from github) and the
multiprocessing module (from python 2.6+).  To run it, start one copy
of fdserver.py and one or more copies of testserver.py (in different
terminals, or backgrounded, etc).  Fetch http://localhost:8000 and
you'll see the requests getting answered by different processes (it's
normal for several requests to go to the same process under light
load, but under heavier load it tends to even out).