Skip to content

Instantly share code, notes, and snippets.

View zsxwing's full-sized avatar
:octocat:

Shixiong Zhu zsxwing

:octocat:
  • Databricks, Inc.
  • San Francisco
View GitHub Profile
@zsxwing
zsxwing / usage.sh
Last active December 15, 2015 21:58
Convenient way to write a usage function.
usage() {
cat <<END;
ez_mail TITLE CONTENT FROM_ADDRESS RECIEVER_ADDRESS
EXAMPLE
ez_mail 'Hello World!' 'Hi!' example@example.com somebody@example.com
END
}
@zsxwing
zsxwing / comment.sh
Created April 7, 2013 12:22
How to use comments in shell.
# one line comment
:<<test
Multiple lines comment.
test
@zsxwing
zsxwing / CurryTest.scala
Created April 8, 2013 02:43
Currying example.
object CurryTest {
def modN(n: Int)(x: Int) = x % n == 0 //> modN: (n: Int)(x: Int)Boolean
val nums = List(1, 2, 3, 4, 5, 6, 7, 8) //> nums : List[Int] = List(1, 2, 3, 4, 5, 6, 7, 8)
println(nums.filter(modN(2))) //> List(2, 4, 6, 8)
println(nums.filter(modN(3))) //> List(3, 6)
}
@zsxwing
zsxwing / Fibonacci.scala
Created April 10, 2013 07:16
Fibonacci Numbers in Scala
import scala.math.BigInt
lazy val fibs: Stream[BigInt] = BigInt(0) #::
BigInt(1) #::
fibs.zip(fibs.tail).map { n => n._1 + n._2 }
@zsxwing
zsxwing / notes.txt
Created August 5, 2013 06:58 — forked from nipra/notes.txt
# Installing CDH4 on a Single Linux Node in Pseudo-distributed Mode
# https://ccp.cloudera.com/display/CDH4DOC/Installing+CDH4+on+a+Single+Linux+Node+in+Pseudo-distributed+Mode
# Installing CDH4 with MRv1 on a Single Linux Node in Pseudo-distributed mode
# On Ubuntu and other Debian systems
nipra@lambda:Downloads$ wget -cv http://archive.cloudera.com/cdh4/one-click-install/precise/amd64/cdh4-repository_1.0_all.deb
nipra@lambda:Downloads$ sudo dpkg -i cdh4-repository_1.0_all.deb # Adds /etc/apt/sources.list.d/cloudera-cdh4.list ??
nipra@lambda:Downloads$ dpkg -L cdh4-repository # To view the files on Ubuntu systems
# Install CDH4
@zsxwing
zsxwing / create.sql
Created August 6, 2013 03:13
Hive - create external table
CREATE EXTERNAL TABLE IF NOT EXISTS access (
ip STRING)
ROW FORMAT DELIMITED FIELDS TERMINATED BY ','
LOCATION '/user/mstr/access';
@zsxwing
zsxwing / exit_when_error.sh
Created September 2, 2013 11:08
We can add -e at the end of "#!/bin/bash" or use "set -e" to let the script exit when some error happens
#!/bin/bash -e
set -e
@zsxwing
zsxwing / build.gradle
Created September 7, 2013 09:40
A template of build.gradle for scala
apply plugin: 'scala'
apply plugin: 'eclipse'
sourceCompatibility = JavaVersion.VERSION_1_6
targetCompatibility = JavaVersion.VERSION_1_6
eclipse {
classpath {
downloadSources = true
downloadJavadoc = false
@zsxwing
zsxwing / async.scala
Last active January 2, 2016 13:39
Asnyc exmaple
import rx.lang.scala.Observable._
import rx.lang.scala.JavaConversions._
import rx.lang.scala.ImplicitFunctionConversions._
import rx.util.async.Async
import java.util.concurrent.Executors
import rx.schedulers.Schedulers
object Test extends App {
val executor = Executors.newFixedThreadPool(10)
@zsxwing
zsxwing / StreamingApp.scala
Last active February 29, 2016 20:51
StreamingApp.scala
package streaming.app
import java.util.UUID
import scala.util.Random
import scala.util.control.NonFatal
import org.apache.commons.io.IOUtils
import org.apache.hadoop.fs.{Path, FileSystem}
import org.apache.spark.sql.SQLContext