This gist consists of Spark presentation examples.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
./configure \ | |
--enable-perlinterp \ | |
--enable-pythoninterp \ | |
--enable-rubyinterp \ | |
--enable-luainterp \ | |
--enable-fail-if-missing \ | |
--enable-cscope \ | |
--enable-gui=auto \ | |
--enable-gtk2-check \ | |
--enable-gnome-check \ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
https://confluence.atlassian.com/display/JIRA/Logging+Work+on+an+Issue#LoggingWorkonanIssue-Loggingworkwhileresolvingorclosinganissue | |
https://confluence.atlassian.com/display/JIRAKB/Set+Remaining+Estimate+to+0+on+post+function |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
List of currenies is generated using python. | |
Dependencies | |
============ | |
pip install pycountry pytz requests |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
function deque_init(d) {d["+"] = d["-"] = 0} | |
function deque_is_empty(d) {return d["+"] == d["-"]} | |
function deque_push_back(d, val) {d[d["+"]++] = val} | |
function deque_push_front(d, val) {d[--d["-"]] = val} | |
function deque_back(d) {return d[d["+"] - 1]} | |
function deque_front(d) {return d[d["-"]]} | |
function deque_pop_back(d) {if(deque_is_empty(d)) {return NULL} else {i = --d["+"]; x = d[i]; delete d[i]; return x}} | |
function deque_pop_front(d) {if(deque_is_empty(d)) {return NULL} else {i = d["-"]++; x = d[i]; delete d[i]; return x}} | |
function deque_print(d){x="["; for (i=d["-"]; i<d["+"] - 1; i++) x = x d[i]", "; print x d[d["+"] - 1]"]; size: "d["+"] - d["-"] " [" d["-"] ", " d["+"] ")"} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
OIFS=$IFS; | |
IFS=","; | |
# fill in your details here | |
dbname=DBNAME | |
user=USERNAME | |
pass=PASSWORD | |
host=HOSTNAME:PORT | |
# first get all collections in the database |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The /user/ directory is owned by "hdfs" with 755 permissions. As a result only hdfs can write to that directory. Unlike unix/linux, hdfs is the superuser and not root. So you would need to do this: | |
sudo -u hdfs hadoop fs -mkdir /user/,,myfile,, | |
sudo -u hdfs hadoop fs -put myfile.txt /user/,,/,, | |
If you want to create a home directory for root so you can store files in his directory, do: | |
sudo -u hdfs hadoop fs -mkdir /user/root | |
sudo -u hdfs hadoop fs -chown root /user/root |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
class Graph[T] { | |
type Vertex = T | |
type GraphMap = Map[Vertex,List[Vertex]] | |
var g:GraphMap = Map() | |
def BFS(start: Vertex): List[List[Vertex]] = { | |
def BFS0(elems: List[Vertex],visited: List[List[Vertex]]): List[List[Vertex]] = { | |
val newNeighbors = elems.flatMap(g(_)).filterNot(visited.flatten.contains).distinct | |
if (newNeighbors.isEmpty) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/bin/bash | |
FILENAME="/tmp/random-lines.$$.tmp" | |
NUMLINES=10000000 | |
seq -f 'line %.0f' $NUMLINES > $FILENAME; | |
echo "10 random lines with nl:" | |
$(which time) -v nl -ba $filename | sort -r | sed 's/.*[0-9]\t//' | head > /dev/null | |
echo "10 random lines with shuf:" | |
$(which time) -v shuf $FILENAME -n10 | head > /dev/null |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
df1.unionAll(df2.select(fd1.columns.map(df1(_)): _*)) |
OlderNewer