Skip to content

Instantly share code, notes, and snippets.

View asmaier's full-sized avatar

asmaier

20 followers · 11 following
View GitHub Profile
@asmaier
asmaier / KafkaProducerIT.java
Last active March 23, 2022 11:16
Simple java junit test of an apache kafka producer (works with Kafka 0.11.0.2) (see also https://github.com/asmaier/mini-kafka)
import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.util.Arrays;
import java.util.Iterator;
import java.util.Properties;
import org.I0Itec.zkclient.ZkClient;
import org.apache.kafka.clients.consumer.ConsumerRecord;
import org.apache.kafka.clients.consumer.ConsumerRecords;
import org.apache.kafka.clients.consumer.KafkaConsumer;
@asmaier
asmaier / ShowSystemProperties.java
Created April 8, 2014 13:33
Small java program showing a sorted list of all java system properties. Useful for debugging, e.g. encoding issues. Compile and run with "javac ShowSystemProperties.java; java ShowSystemProperties".
import java.util.Map;
import java.util.Properties;
import java.util.TreeMap;
/**
* Shows a sorted list of all system properties
*/
public class ShowSystemProperties {
public static void main(String[] argv) {
@asmaier
asmaier / wir-sind-bild.ipynb
Last active August 29, 2015 14:25
Unsmoothed Maximum Likelihood Character Level Language Model to create headlines for the german Bild-Zeitung
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@asmaier
asmaier / hdp26-spark2-livy-cluster.json
Created July 14, 2017 15:08
Ambari blueprint for Hortonworks 2.6 with Spark 2.1 and Livy Server (see https://community.hortonworks.com/articles/104448/using-cloudbreak-to-deploy-hdp-26-and-spark-21-on.html)
{
"host_groups": [
{
"name": "host_group_client_1",
"configurations": [],
"components": [
{
"name": "ZOOKEEPER_CLIENT"
},
{
@asmaier
asmaier / start_cloudera.sh
Created July 17, 2017 16:19
Start script for cloudera quickstart docker container (see https://dzone.com/articles/docker-how-to-get-started-with-cloudera)
#!/bin/bash
docker run --name cdh --hostname "quickstart.cloudera" --privileged=true -t -i -d \
-p 80:80 \
-p 7180:7180 \
-p 8888:8888 \
cloudera/quickstart /usr/bin/docker-quickstart
docker exec -ti cdh /home/cloudera/cloudera-manager --express
@asmaier
asmaier / load_parquet_s3.py
Last active March 5, 2021 03:43
Pyspark script for downloading a single parquet file from Amazon S3 via the s3a protocol. It also reads the credentials from the "~/.aws/credentials", so we don't need to hardcode them. See also https://hadoop.apache.org/docs/stable/hadoop-aws/tools/hadoop-aws/index.html .
#
# Some constants
#
aws_profile = "your_profile"
aws_region = "your_region"
s3_bucket = "your_bucket"
#
# Reading environment variables from aws credential file
#
@asmaier
asmaier / gist:1e38c07464a2cf50e7180b159f74faa8
Last active October 17, 2017 15:25
Investigate political landscape (using Wahl-O-Mat data)
@asmaier
asmaier / python_type_hints.md
Last active January 16, 2018 17:18
Minimal overview of Python Type Hints

Python Type Hints

  • Type after variable
  • Problem: IDE cannot suggest variable name based on type

Primitives

From

// primitive function
@asmaier
asmaier / install_m2crypto.md
Last active May 18, 2020 17:01 — forked from andrisasuke/install_m2crypto.txt
python install m2crypto on Mac OS X

Do

$> brew install openssl
$> brew install swig

Now add these lines to your ~/.zshrc file on Mac OS X

# this is very slow, see https://github.com/Homebrew/brew/issues/3327
# OPENSSL=$(brew --prefix openssl)
# this is much faster
@asmaier
asmaier / latex2tufte.md
Last active November 11, 2022 21:13
Converting latex files into HTML5 with Tufte Css

Latex to Tufte-HTML

To convert LaTeX to HTML5 with tufte-css layout one first needs to copy the file tufte.css and the corresponding font directory et-book from the tufte-css github repo into your working directory.

Then we need to extract the default html5 template from pandoc via

pandoc -D html5 > html5.html

This template must be modified and the tags <article> and <section> should be added around the body of the template like