Skip to content

Instantly share code, notes, and snippets.

View huangjs's full-sized avatar

Jianshi Huang huangjs

View GitHub Profile
@huangjs
huangjs / 0_reuse_code.js
Created October 20, 2015 03:19
Here are some things you can do with Gists in GistBox.
// Use Gists to store code you would like to remember later on
console.log(window); // log the "window" object to the console
// Generated by gfwlist2pac in precise mode
// https://github.com/clowwindy/gfwlist2pac
var proxy = "SOCKS5 127.0.0.1:1080; SOCKS 127.0.0.1:1080; DIRECT;";
var rules = [
"||altrec.com",
"||darpa.mil",
"||fastpic.ru",
"||fxnetworks.com",
@huangjs
huangjs / gist:2c98b71bc912541eec7b
Created April 24, 2015 09:34
Spark Parquet error reading dataset containing array of structs
+-----------+----------------------------+----------+
| col_name | data_type | comment |
+-----------+----------------------------+----------+
| cust_id | string | |
| part_num | int | |
| ip_list | array<struct<ip:string>> | |
| vid_list | array<struct<vid:string>> | |
| fso_list | array<struct<fso:string>> | |
| src | string | |
| date | int | |
@huangjs
huangjs / gist:683a4c85ae14e9ae205b
Created December 13, 2014 01:37
createParquetTable
def createParquetTable(name: String, file: String)(implicit sqlc: HiveContext): Unit = {
import org.apache.spark.sql.hive.HiveMetastoreTypes
import sqlc._
val rdd = parquetFile(file)
val schema = rdd.schema.fields.map(f => s"`${f.name}` ${HiveMetastoreTypes.toMetastoreType(f.dataType)}").mkString(",\n")
val cleanup = s"""
|DROP TABLE IF EXISTS $name
""".stripMargin
@huangjs
huangjs / gist:4273403
Created December 13, 2012 01:54
Example code for using FullContact influencer notification in iced-coffee-script on node.js.
# This is example code for influencer notification using FullContact on node.js
# Programming language used here is iced-coffee-script
# Author: Maptia
# License: public domain
_ = require 'underscore'
iced = require('iced-coffee-script').iced
request = require 'request'
fullContactAPIKey = '<your-fullcontact-api-key>'
@huangjs
huangjs / gist:3866151
Created October 10, 2012 14:57
query-prepared and flet-prepared for postmodern
(defmacro query-preparing (query &rest args/format)
"Like query, but make the query prepared (only once)."
(let* ((format :rows)
(args (loop for arg in args/format
if (or (postmodern::dao-spec-for-format arg)
(assoc arg postmodern::*result-styles*))
do (setf format arg)
else
collect arg)))
`(funcall
@huangjs
huangjs / gist:3764039
Created September 21, 2012 21:30
Nominatim installation diagnosis
..............................................................................................................................................................................................................................................................
Reanalysing database...
NOTICE: no notnull values, invalid stats
NOTICE: no notnull values, invalid stats
ANALYZE
ERROR: please download osmosis
please download osmosis
@huangjs
huangjs / gist:2952481
Created June 19, 2012 05:45
socketstream with HTTPS doesn't work
var port = process.env.HTTP_PORT || 3000
, securePort = process.env.HTTPS_PORT || 3443
var httpsOptions = {
key: fs.readFileSync('./cert/private.key.insecure').toString()
, cert: fs.readFileSync('./cert/public.cert').toString()
};
// Start web server
var server = http.createServer(ss.http.middleware)
CL-USER> (defvar *a* 1)
*A*
CL-USER> (let ((*a* 2)) (sb-thread:make-thread (lambda () (print *a*))))
1
#<SB-THREAD:THREAD FINISHED values: 1 {100482B0C1}>
CL-USER> (let ((*a* 2)) (when (zerop (sb-posix:fork)) (with-open-file (f "/tmp/a-value" :direction :output) (print *a* f))))
NIL
$ cat /tmp/a-value