Skip to content

Instantly share code, notes, and snippets.

View leetschau's full-sized avatar

Leo leetschau

26 followers · 8 following
View GitHub Profile
@leetschau
leetschau / install redmine with puppet
Created February 11, 2014 03:15
install redmine with puppet
Today I try to install Redmine server with Puppet on a vagrant managed virtual machine. I use a published module on puppet forge:
http://forge.puppetlabs.com/johanek/redmine
And the following is my operations:
1. Initialize VM:
chad@typcserver ~/docs/vagrant-prj $ vagrant --version
Vagrant 1.4.3
chad@typcserver ~/docs/vagrant-prj $ vagrant init
chad@typcserver ~/docs/vagrant-prj $ cat Vagrantfile
@leetschau
leetschau / gist:42347eaf142c780b0fa0
Created November 27, 2014 02:45
Spark record delimiter definition
When I run above codes in spark-shell, I got the following errors:
scala> val job = new Job(sc.hadoopConfiguration)
warning: there were 1 deprecation warning(s); re-run with -deprecation for details
java.lang.IllegalStateException: Job in state DEFINE instead of RUNNING
at org.apache.hadoop.mapreduce.Job.ensureState(Job.java:283)
at org.apache.hadoop.mapreduce.Job.toString(Job.java:452)
at scala.runtime.ScalaRunTime$.scala$runtime$ScalaRunTime$$inner$1(ScalaRunTime.scala:324)
at scala.runtime.ScalaRunTime$.stringOf(ScalaRunTime.scala:329)
at scala.runtime.ScalaRunTime$.replStringOf(ScalaRunTime.scala:337)
@leetschau
leetschau / progress-button.html
Created May 20, 2016 05:19
Progress Button Group
<html><br><head><br><link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.6/css/bootstrap.min.css" integrity="sha384-1q8mTJOASx8j1Au+a5WDVnPi2lkFfwwEAa8hDDdjZlpLegxhjVME1fgjWPGmkzs7" crossorigin="anonymous"><br><br><style><br> .progress-group {<br> margin: 50px;<br> width: 50%;<br> }<br> .progress-button {<br> position: relative;<br> }<br> .progress-button::before {<br> position: absolute;<br> border-width: 17px;<br> border-style: solid;<br> content: "";<br> right: -34px; <br> top: -1px;<br> border-color: transparent transparent transparent #ccc;<br> z-index: 50;<br> }<br> .progress-button::after {<br> position: absolute;<br> border-width: 16px;<br> border-style: solid;<br> content: "";<br> right: -32px; <br> top: 0;<br> border-color: transparent transparent transparent white;<br> z-index: 50;<br> }<br> .progress-button.btn-lg::before {<br> border-width: 23px;<br> right: -46px; <br> }<br> .progress-button.btn-lg::af
@leetschau
leetschau / convertDate.sh
Last active May 28, 2016 12:14
MongoDB导出的json数据中，时间字段前面都有{$date: 2016.04.04}，$date不能被Elasticsearch识别，需要去掉，从{"timeStart: {$date: "xxx"}}变成{"timeStart: "xxx"}
#!/bin/bash
sed 's/{\"\$date\":\(\".\{10\}T.\{12\}Z\"\)}/\1/g' f1000.json > temp.json
sed -i 25d temp.json
mongoimport -d test -c strDate --type json --file temp.json
# sample data:
"timeStart":{"$date":"1015-09-01T00:00:00.000Z"},"timeEnd":{"$date":"2018-03-02T00:00:00.000Z"},
"timeStart":{"$date":"2016-09-01T00:00:00.000Z"},"timeEnd":{"$date":"2021-04-0kT00:00:00.000Z"},
"timeStart":{"$date":"2017-09-01T00:00:00.000Z"},"timeEnd":{"$date":"2021-05-09T00:00:00.000Z"},
@leetschau
leetschau / sorted.py
Created May 28, 2016 13:54
Python sorted 函数，当某项不包含排序依据时，可以将fallback值作为自己的排序依据
aa = [{'a': 3}, {'a': 2}, {'a': 5}, {'b':8}]
sorted(aa, key=lambda rec: rec['a'] if 'a' in rec else 1, reverse=True)
#: [{'a': 5}, {'a': 3}, {'a': 2}, {'b': 8}]
sorted(aa, key=lambda rec: rec['a'] if 'a' in rec else 6, reverse=True)
#: [{'b': 8}, {'a': 5}, {'a': 3}, {'a': 2}]
@leetschau
leetschau / cpColl.sh
Created May 28, 2016 14:06
Copy collection between MongoDB servers
#!/usr/bin/python
import sys, re
from os import system
usage = """
Usage:
Full style: ./cpColl sUser:sPwd@sHost:sPort/sDb-sColl dUser:dPwd@dHost:dPort/dDb-dColl
Abbr for source on localhost: ./cpColl sDb-sColl dUser:dPwd@dHost:dPort/dDb-dColl
Abbr for destination on localhost: ./cpColl sUser:sPwd@sHost:sPort/sDb-sColl dDb-dColl
@leetschau
leetschau / multi-dimension-correlation-matrix.py
Last active July 7, 2016 03:26
在Anaconda环境下运行，详细分析见"Data Science from Scratch"第10章读书笔记
import math
import random
import numpy as np
import matplotlib.pyplot as plt
def normal_cdf(x, mu=0, sigma=1):
return (1 + math.erf((x - mu) / math.sqrt(2) / sigma)) / 2
@leetschau
leetschau / arguments_parser.py
Last active September 25, 2016 00:29
import argparse
parser = argparse.ArgumentParser(
description='Sync docs in MongoDB to Elasticsearch server')
parser.add_argument('inputfile_name',
help='the json file contains data to synced from MongoDB')
parser.add_argument('-s', '--server-name',
help='the IP address of Elasticsearch server')
parser.add_argument('-i', '--index', help='the index name of ES')
parser.add_argument('-t', '--type-name', help='the type name of the index')
@leetschau
leetschau / dsnote2evernote.py
Created September 17, 2016 10:06
将dsnote里面的笔记通过`geeknote create`命令同步到Evernote上
import os
import subprocess
basedir = '/home/leo/.donno/repo'
notebook = 'dsnote'
err_coll = []
for fn in os.listdir(basedir):
print('upload %s' % fn)
fullname = os.path.join(basedir, fn)
@leetschau
leetschau / docker-show-repo-tags.sh
Created September 26, 2016 08:45
list all tags of an image in docker hub
#!/bin/sh
# From: http://stackoverflow.com/a/34054903/701420
# Simple script that will display docker repository tags.
#
# Usage:
# $ docker-show-repo-tags.sh ubuntu centos
for Repo in $* ; do
curl -s -S "https://registry.hub.docker.com/v2/repositories/library/$Repo/tags/" | \
sed -e 's/,/,\n/g' -e 's/\[/\[\n/g' | \
grep '"name"' | \