Navigation Menu

Skip to content

Instantly share code, notes, and snippets.

View zsxwing's full-sized avatar
:octocat:

Shixiong Zhu zsxwing

:octocat:
999 followers · 29 following
  • Databricks, Inc.
  • San Francisco
View GitHub Profile
@zsxwing
zsxwing / argument.sh
Last active December 15, 2015 21:58
Set the $1 $2 $3, ..., and $# in the current terminal. Avoid to open a new file when you just want to test a small function.
set -- a b c
echo "$1 $2 $3"
@zsxwing
zsxwing / iter_lines.sh
Last active December 15, 2015 21:58
How to iterate lines in a file or generated by a command correctly.
while read line; do
echo $line;
done < <(cat test.txt)
@zsxwing
zsxwing / long_string.sh
Last active December 15, 2015 21:58
How to create a long string in bash shell
# output "I am an exceptionally long string."
long_string="I am an exceptionally
long string."
echo ${long_string}
@zsxwing
zsxwing / error_log.sh
Last active December 15, 2015 21:58
A error function to output the message with time.
err() {
echo "[$(date +'%Y-%m-%dT%H:%M:%S%z')]: $@" >&2
}
@zsxwing
zsxwing / usage.sh
Last active December 15, 2015 21:58
Convenient way to write a usage function.
usage() {
cat <<END;
ez_mail TITLE CONTENT FROM_ADDRESS RECIEVER_ADDRESS
EXAMPLE
ez_mail 'Hello World!' 'Hi!' example@example.com somebody@example.com
END
}
@zsxwing
zsxwing / comment.sh
Created April 7, 2013 12:22
How to use comments in shell.
# one line comment
:<<test
Multiple lines comment.
test
@zsxwing
zsxwing / CurryTest.scala
Created April 8, 2013 02:43
Currying example.
object CurryTest {
def modN(n: Int)(x: Int) = x % n == 0 //> modN: (n: Int)(x: Int)Boolean
val nums = List(1, 2, 3, 4, 5, 6, 7, 8) //> nums : List[Int] = List(1, 2, 3, 4, 5, 6, 7, 8)
println(nums.filter(modN(2))) //> List(2, 4, 6, 8)
println(nums.filter(modN(3))) //> List(3, 6)
}
@zsxwing
zsxwing / Fibonacci.scala
Created April 10, 2013 07:16
Fibonacci Numbers in Scala
import scala.math.BigInt
lazy val fibs: Stream[BigInt] = BigInt(0) #::
BigInt(1) #::
fibs.zip(fibs.tail).map { n => n._1 + n._2 }
@zsxwing
zsxwing / gist:5924191
Last active April 3, 2017 04:45
Pig and Hive source codes overview. Small hint: use "explain" to find the operation keywords, and search them in the source codes to find the corresponding class.
Pig:
org.apache.pig.newplan.logical.relational.LogicalPlan
org.apache.pig.backend.hadoop.executionengine.physicalLayer.plans.PhysicalPlan
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.plans.MROperPlan
org.apache.pig.parser.QueryParserDriver.parse(String)
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.compile(LogicalPlan,Properties)
org.apache.pig.PigServer.launchPlan(PhysicalPlan,String)
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.compile(PhysicalPlan,PigContext)
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.getJob(MROperPlan,MapReduceOper,Configuration,PigContext)
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(MROperPlan,String)
@zsxwing
zsxwing / notes.txt
Created August 5, 2013 06:58 — forked from nipra/notes.txt
# Installing CDH4 on a Single Linux Node in Pseudo-distributed Mode
# https://ccp.cloudera.com/display/CDH4DOC/Installing+CDH4+on+a+Single+Linux+Node+in+Pseudo-distributed+Mode
# Installing CDH4 with MRv1 on a Single Linux Node in Pseudo-distributed mode
# On Ubuntu and other Debian systems
nipra@lambda:Downloads$ wget -cv http://archive.cloudera.com/cdh4/one-click-install/precise/amd64/cdh4-repository_1.0_all.deb
nipra@lambda:Downloads$ sudo dpkg -i cdh4-repository_1.0_all.deb # Adds /etc/apt/sources.list.d/cloudera-cdh4.list ??
nipra@lambda:Downloads$ dpkg -L cdh4-repository # To view the files on Ubuntu systems
# Install CDH4