Skip to content

Instantly share code, notes, and snippets.

@fdeantoni
fdeantoni / GBT.scala
Created May 13, 2015 05:06
Spark Gradient Boosted Tree
import org.apache.spark.{SparkContext, SparkConf}
import org.apache.spark.mllib.tree.GradientBoostedTrees
import org.apache.spark.mllib.tree.configuration.BoostingStrategy
import org.apache.spark.mllib.util.MLUtils
object GBT {
def main(args: Array[String]): Unit = {
val conf = new SparkConf(false) // skip loading external settings
@fdeantoni
fdeantoni / ps_logger.sh
Last active February 3, 2016 07:25
ps monitoring
#!/usr/bin/env bash
LOG="/home/allied/ps.log"
nohup ./ps_output.sh > ${LOG} 2>&1 &
@fdeantoni
fdeantoni / install-mesos-dns.sh
Last active October 18, 2016 01:17
Scritps to install Mesos, Marathon, and Mesos DNS on Ubuntu 14.04 Server. The script install-mesos-master.sh and install-mesos-dns.sh are run on the master nodes. The scripts install-mesos-slave.sh and install-mesos-docker.sh are run on the slave nodes.
#!/bin/bash
set -e
# Parametes
ZK_M1=10.10.25.10
ZK_M2=10.10.25.11
ZK_M3=10.10.25.12
ZK_CLUSTER="zk://$ZK_M1:2181,$ZK_M2:2181,$ZK_M3:2181"
# Download mesos dns
wget https://github.com/mesosphere/mesos-dns/releases/download/v0.5.2/mesos-dns-v0.5.2-linux-amd64
chmod +x mesos-dns-v0.5.2-linux-amd64
@fdeantoni
fdeantoni / install.yml
Created September 12, 2019 06:14
Install k3s with Ansible
# First install python on all nodes as ansible needs this
- hosts: all
become: true
gather_facts: false
pre_tasks:
- name: Install python2 for Ansible
raw: bash -c "test -e /usr/bin/python || (apt -qqy update && apt install -qqy python-minimal)"
register: output
changed_when: output.stdout != ""
@fdeantoni
fdeantoni / cluster.yaml
Created September 12, 2019 07:11
Rook on k3s
apiVersion: ceph.rook.io/v1
kind: CephCluster
metadata:
name: rook-ceph
namespace: rook-ceph
spec:
cephVersion:
image: ceph/ceph:v14.2.1-20190430
dataDirHostPath: /var/lib/rook
mon:
@fdeantoni
fdeantoni / docker-entrypoint.sh
Last active September 16, 2019 09:01
Docker entrypoint file to launch NiFi in clustered mode with SSL enabled
#!/bin/bash
if [[ "$1" = 'nifi' ]]; then
scripts_dir='/opt/nifi/scripts'
[[ -f "${scripts_dir}/common.sh" ]] && . "${scripts_dir}/common.sh"
# Override JVM memory settings
if [[ ! -z "${NIFI_JVM_HEAP_INIT}" ]]; then
@fdeantoni
fdeantoni / Dockerfile
Created September 16, 2019 08:57
Custom docker image to run NiFi with SSL in clustered mode on Mesos Marathon
FROM apache/nifi:1.9.2
ARG NAR_VERSION="1.0.0-SNAPSHOT"
ENV NAR_FILE_NAME="custom-processor-${NAR_VERSION}.nar"
ENV NAR_FILE_SOURCE="build/libs/${NAR_FILE_NAME}"
ENV FLOW_FILE_SOURCE="build/libs/flow.xml.gz"
ENV LOGBACK_FILE_SOURCE="build/resources/main/logback.xml"
ENV DRIVERS_SOURCE="build/drivers/*jar"
ENV DRIVERS_TARGET="/home/nifi/jdbc-drivers/"
@fdeantoni
fdeantoni / haproxy.cfg
Last active September 17, 2019 04:59
HAProxy configuration with Prometheus and WebDAV using two-way SSL authentication.
global
log 127.0.0.1 local0 info
spread-checks 5
max-spread-checks 15000
maxconn 50000
tune.ssl.default-dh-param 2048
ssl-default-bind-ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA256:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:!aNULL:!MD5:!DSS
ssl-default-bind-options no-sslv3 no-tlsv10 no-tls-tickets
ssl-default-server-ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA256:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:!aNULL:!MD
@fdeantoni
fdeantoni / sensor-schema.json
Created September 18, 2019 00:33
ElasticSearch schema for storing sensor data with MQTT topic and WKT geometry string
{
"settings": {
"number_of_shards": 1,
"analysis": {
"analyzer": {
"sensor_path_tree": {
"tokenizer": "sensor_hierarchy"
},
"sensor_path_tree_reversed": {
"tokenizer": "sensor_hierarchy_reversed"
@fdeantoni
fdeantoni / OkHttpActor.scala
Created September 18, 2019 02:43
OkHttp with Akka Streaming
import java.nio.ByteBuffer
import java.util.concurrent.TimeUnit
import akka.actor._
import akka.util.ByteString
import okhttp3._
import okio.{ByteString => OkioByteString}
import scala.util.Try