sudo tcptraceroute 140.211.11.121 443
Selected device eth0, address 188.165.250.161, port 38659 for outgoing packets
Tracing the path to 140.211.11.121 on TCP port 443 (https), 30 hops max
1 vss-4-6k.fr.eu (188.165.250.253) 121.387 ms 1.456 ms *
2 rbx-g1-a9.fr.eu (94.23.122.161) 0.609 ms 0.601 ms 0.545 ms
3 th2-g1-a9.fr.eu (91.121.131.210) 4.539 ms 4.293 ms 4.071 ms
4 * * *
If you're as lazy as I am for your cloud based pet projects, use this simple ansible play to setup a DNS-ready infrastructure :)
- lineinfile: dest=/etc/hosts regexp=".*{{ item }}.*" line="{{ hostvars[item].ansible_default_ipv4.address }} {{ item }}" state=present
sudo: yes
with_items: hostvars | list
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# hadoop classpath prior to 2.6 returns wildcarded path (starting hadoop 2.6, hadoop classpath --glob is doing the trick by itself) | |
# It's sometimes convenient to have a nicely printed list of jars that that part of the classpath when running hadoop jar. | |
for i in $(hadoop classpath | sed -e "s/:/ /g"); | |
do | |
echo $i | egrep "\.jar$" >/dev/null && python -c "import os,sys; print os.path.realpath(sys.argv[1])" $i | |
done | sort | uniq | |
# If you want to generate the list of ArtifactIds to exclude from your packaging, use this version |
This is a "one liner" shell command to list all partitions with not in-synch replicas (i.e. with a list of in-synch replicas not matching the list of replicas).
Please set ZK
variable accordingly.
/usr/hdp/current/kafka-broker/bin/kafka-topics.sh --zookeeper $ZK --describe | grep "Isr: " | while read line;
do
replicas=$(echo $line | sed -e "s/.*Replicas: \([^ ]*\).*/\1/" -e "s/,/\n/g" | sort -n | paste -sd ",");
isr=$(echo $line | sed -e "s/.*Isr: \([^ ]*\).*/\1/" -e "s/,/\n/g" | sort -n | paste -sd ",");
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ls -1 /var/run/cloudera-scm-agent/process/ | while read line; do id=$(echo $line | cut -d "-" -f1); process=$(echo $line | sed -e "s/^[0-9]*-//"); echo "$process $id"; done | sort -k1,1r -k2,2nr | while read key value; do if [ "$cur_key" = "$key" ]; then echo -n ",$value"; else if [ "" != "$cur_key" ]; then echo; fi; echo -n "$key:$value"; cur_key=$key; fi; done | grep "." | while read line; do process=$(echo $line | cut -d ":" -f1); ids=$(echo $line | cut -d ":" -f2 | sed -e "s/,/ /g"); first=0; for i in $ids; do if [ "0" = "$first" ]; then first=1; else echo "Deleting /var/run/cloudera-scm-agent/process/${i}-${process}"; rm -fr "/var/run/cloudera-scm-agent/process/${i}-${process}"; fi; done; done |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
diff --git a/tez-mapreduce/src/main/java/org/apache/tez/mapreduce/hadoop/mapreduce/JobContextImpl.java b/tez-mapreduce/src/main/java/org/apache/tez/mapreduce/hadoop/mapreduce/JobContextImpl.java | |
index 12491ed..b4ca24c 100644 | |
--- a/tez-mapreduce/src/main/java/org/apache/tez/mapreduce/hadoop/mapreduce/JobContextImpl.java | |
+++ b/tez-mapreduce/src/main/java/org/apache/tez/mapreduce/hadoop/mapreduce/JobContextImpl.java | |
@@ -475,5 +475,16 @@ public class JobContextImpl implements JobContext { | |
public Progressable getProgressible() { | |
return progress; | |
} | |
+ | |
+ /** |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
diff --git a/pom.xml b/pom.xml | |
index 1e29c4d..30aef6a 100644 | |
--- a/pom.xml | |
+++ b/pom.xml | |
@@ -1048,6 +1048,27 @@ | |
</plugins> | |
</build> | |
</profile> | |
+ <profile> | |
+ <id>cdh5.2</id> |