fillest

# Bash really should be avoided as much as possible (within reasonable limits, of course) even for one-liners which *seem* trivial.
# Bash is very error-prone by design. It's hard to comprehend all the pitfalls (e.g. https://mywiki.wooledge.org/BashFAQ/105)
# and it's a regrettable time-waste anyway.
#
# Modern Python is good for scripting the logic - keep Bash only for launching executables and most primitive
# pipes and redirections (avoid subshells, substitutions and so on). No need to install anything -
# just start your script with the following small self-contained helper function (check the examples for usage). Its features:
# * terminates on non-zero exit status by default
# * returns the output (combined - which usually should not be a problem - use e.g. '2>/dev/null' when it is)
# * prints commands and combined output

mholtzscher

from airflow.models import DAG
from airflow.operators.email_operator import EmailOperator
from airflow.operators.python_operator import PythonOperator
from datetime import datetime
from tempfile import NamedTemporaryFile


dag = DAG(
    "email_example",
    description="Sample Email Example with File attachments",

djspiewak

Quick Tips for Fast Code on the JVM

I was talking to a coworker recently about general techniques that almost always form the core of any effort to write very fast, down-to-the-metal hot path code on the JVM, and they pointed out that there really isn't a particularly good place to go for this information. It occurred to me that, really, I had more or less picked up all of it by word of mouth and experience, and there just aren't any good reference sources on the topic. So… here's my word of mouth.

This is by no means a comprehensive gist. It's also important to understand that the techniques that I outline in here are not 100% absolute either. Performance on the JVM is an incredibly complicated subject, and while there are rules that almost always hold true, the "almost" remains very salient. Also, for many or even most applications, there will be other techniques that I'm not mentioning which will have a greater impact. JMH, Java Flight Recorder, and a good profiler are your very best friend! Mea

elmodaddyb

Load Gerbera as a System Daemon

This readme outlines how to add the Gerbera runtime as a system daemon using the systemd.

Prerequisites

You installed gerbera to /usr/local/bin/gerbera

If you don't know the path try which gerbera

pheymann

import shapeless._
import shapeless.labelled.FieldType
import shapeless.record._

trait GenericDiff[H <: HList] {

  // syntactic sugar
  type HI = H

  // compares field values and returns the field name with values if they differ

d1egoaz

If you want to learn Scala, I’d suggest:

• Install IntelliJ + Scala Plugin
• Don’t do the Coursera courses yet.
• Don’t do the “red book” Functional Programming in Scala yet.
• Do: http://underscore.io/books/
  • Essential Scala
  • Essential Play
  • Essential Slick
• Do Scala for the Impatient: https://www.amazon.com/Scala-Impatient-Cay-S-Horstmann/dp/0321774094
• The Neophyte’s Guide to Scala http://danielwestheide.com/scala/neophytes.html

timvw

case class RemoteIteratorWrapper[T](underlying: org.apache.hadoop.fs.RemoteIterator[T]) extends scala.collection.AbstractIterator[T] with scala.collection.Iterator[T] {
  def hasNext = underlying.hasNext
  def next() = underlying.next()
}

object Conversions {
  implicit def remoteIterator2ScalaIterator[T](underlying: org.apache.hadoop.fs.RemoteIterator[T]) : scala.collection.Iterator[T] = RemoteIteratorWrapper[T](underlying)
}

staltz

The introduction to Reactive Programming you've been missing

(by @andrestaltz)

---

This tutorial as a series of videos

If you prefer to watch video tutorials with live-coding, then check out this series I recorded with the same contents as in this article: Egghead.io - Introduction to Reactive Programming.

---

roxberry

hadoop fs -ls albumrelease-tracks/output/ | awk '{print $8}' | \
while read f
do
  hadoop fs -cat $f | grep -q MT0002267152 && echo $f
done

jboner

Latency Comparison Numbers (~2012)
----------------------------------
L1 cache reference                           0.5 ns
Branch mispredict                            5   ns
L2 cache reference                           7   ns                      14x L1 cache
Mutex lock/unlock                           25   ns
Main memory reference                      100   ns                      20x L2 cache, 200x L1 cache
Compress 1K bytes with Zippy             3,000   ns        3 us
Send 1K bytes over 1 Gbps network       10,000   ns       10 us
Read 4K randomly from SSD*             150,000   ns      150 us          ~1GB/sec SSD