Skip to content

Instantly share code, notes, and snippets.

@ottomata
Created January 23, 2013 18:00
Show Gist options
  • Save ottomata/4611076 to your computer and use it in GitHub Desktop.
Save ottomata/4611076 to your computer and use it in GitHub Desktop.
mobile webrequest udp2log vs hdfs
for host in cp104{1,2,3,4}; do
first_seq=$(head -n 1 $host.udp2log.seq | awk '{print $1}') # && echo $first_seq
last_seq=$(tail -n 1 $host.hdfs.seq | awk '{print $1}') # && echo $last_seq
should_be=$(echo "$last_seq - $first_seq + 1" | bc) #&& echo $should_be
# count lines in blog.hdfs.seq file between $first_seq and $last_seq
hdfs_count=$(sed -n -e "/^$first_seq/,/^$last_seq/p" $host.udp2log.seq | wc -l) #&& echo $hdfs_count
udp2log_count=$(sed -n -e "/^$first_seq/,/^$last_seq/p" $host.udp2log.seq | wc -l) #&& echo $udp2log_count
echo "----"
echo "$host:"
echo " should_be: $should_be"; echo " udp2log_count: $udp2log_count"; echo " hdfs_count: $hdfs_count";
done
----
cp1041:
should_be: 2996340
udp2log_count: 2558004
hdfs_count: 2558004
----
cp1042:
should_be: 2992324
udp2log_count: 2554088
hdfs_count: 2554088
----
cp1043:
should_be: 3004241
udp2log_count: 2565111
hdfs_count: 2565111
----
cp1044:
should_be: 3005422
udp2log_count: 2566392
hdfs_count: 2566392
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment