rmoff

---
version: '2'
services:
  zookeeper:
    image: confluentinc/cp-zookeeper:latest
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
      ZOOKEEPER_TICK_TIME: 2000

  kafka:

davideicardi

Avro serialization

There are 4 possible serialization format when using avro:

• Avro Json encoding
• Avro Data Serialization (https://avro.apache.org/docs/current/spec.html#Data+Serialization) Binary format with an header that contains the full schema, this is the format usually used when writing Avro files
• Avro Single Object Encoding (https://avro.apache.org/docs/current/spec.html#single_object_encoding) Binary format with an header with only the fingerprint/id of the schema, this it the format used by Kafka (see this
• Avro Binary Encoding (https://avro.apache.org/docs/current/spec.html#binary_encoding)

weaming

#!/usr/bin/env python3
# coding: utf-8
"""
Author       : weaming
Created Time : 2018-05-26 21:32:59
Prerequisite:
    python3 -m pip install cson arrow
"""
import json
import os

marwei

Kafka 0.11.0.0 (Confluent 3.3.0) added support to manipulate offsets for a consumer group via cli kafka-consumer-groups command.

1. List the topics to which the group is subscribed

kafka-consumer-groups --bootstrap-server <kafkahost:port> --group <group_id> --describe

Note the values under "CURRENT-OFFSET" and "LOG-END-OFFSET". "CURRENT-OFFSET" is the offset where this consumer group is currently at in each of the partitions.

2. Reset the consumer offset for a topic (preview)

yoyama

/**
  * Generate Case class from DataFrame.schema
  *
  *  val df:DataFrame = ...
  *
  *  val s2cc = new Schema2CaseClass
  *  import s2cc.implicit._
  *
  *  println(s2cc.schemaToCaseClass(df.schema, "MyClass"))
  *

kyrsideris

"""
This script exercises the join as well as left, right and full outer join as implemented in
Apache Spark. Employee and Department tables were inspired by the examples in wiki's articles
on join operation: https://en.wikipedia.org/wiki/Join_(SQL)


Employee
    (31, 'Rafferty')
    (33, 'Jones')
    (33, 'Heisenberg')

protrolium

ffmpeg

Converting Audio into Different Formats / Sample Rates

Minimal example: transcode from MP3 to WMA:
ffmpeg -i input.mp3 output.wma

You can get the list of supported formats with:
ffmpeg -formats

You can get the list of installed codecs with:

delagoya

@namespace("com.example")
/**
This is a comment for the whole protocol
*/

protocol Example {

/**

The comment applies to the `NoSpaces` record, but is not indented to the 

amscotti

crypto = require('crypto');

#Quick MD5 of text
text = "MD5 this text!"
md5hash1 = crypto.createHash('md5').update(text).digest("hex")

#MD5 of text with updates
m = crypto.createHash('md5') 
m.update("MD5 ")
m.update("this ")