aclisp

# Install Java 1.8 in CentOS/RHEL 6.X
sudo yum remove -y java-1.6.0-openjdk
wget --no-cookies \
--no-check-certificate \
--header "Cookie: oraclelicense=accept-securebackup-cookie" \
"http://download.oracle.com/otn-pub/java/jdk/8-b132/jdk-8-linux-x64.rpm" \
-O jdk-8-linux-x64.rpm
sudo rpm -Uvh jdk-8-linux-x64.rpm
sudo alternatives --install /usr/bin/java java /usr/java/latest/jre/bin/java 20000
sudo alternatives --install /usr/bin/jar jar /usr/java/latest/bin/jar 20000

aclisp

# check if job exists
curl -XGET 'http://jenkins/checkJobName?value=yourJobFolderName' --user user.name:YourAPIToken

# with folder plugin
curl -s -XPOST 'http://jenkins/job/FolderName/createItem?name=yourJobName' --data-binary @config.xml -H "Content-Type:text/xml" --user user.name:YourAPIToken

# without folder plugin
curl -s -XPOST 'http://jenkins/createItem?name=yourJobName' --data-binary @config.xml -H "Content-Type:text/xml" --user user.name:YourAPIToken

# create folder

aclisp

Easiest HDFS cluster in the world with kubernetes.

Inspiration from kimoonkim/kubernetes-HDFS

kubectl create -f namenode.yaml
kubectl create -f datanode.yaml

Setup a port-forward to so you can see it is alive:

aclisp

package gziphandler

import (
	"compress/gzip"
	"log"
	"net/http"
	"strings"
	"sync"
)

aclisp

HTTP Streaming (or Chunked vs Store & Forward)

The standard way of understanding the HTTP protocol is via the request reply pattern. Each HTTP transaction consists of a finitely bounded HTTP request and a finitely bounded HTTP response.

However it's also possible for both parts of an HTTP 1.1 transaction to stream their possibly infinitely bounded data. The advantages is that the sender can send data that is beyond the sender's memory limit, and the receiver can act on

aclisp

Here's what I ended up doing - it worked perfectly. Note that I was moving from my old host (Bitbucket) to my new one (Gitlab). My comments are above the commands:

# First, shallow-clone the old repo to the depth we want to keep
git clone --depth=50 https://...@bitbucket.org/....git

# Go into the directory of the clone
cd clonedrepo

# Once in the clone's repo directory, remove the old origin
git remote remove origin

aclisp

function hex_dump (str)
    local len = string.len( str )
    local dump = ""
    local hex = ""
    local asc = ""
    
    for i = 1, len do
        if 1 == i % 8 then
            dump = dump .. hex .. asc .. "\n"
            hex = string.format( "%04x: ", i - 1 )