Skip to content

Instantly share code, notes, and snippets.

View drldcsta's full-sized avatar

Darrell DeCosta drldcsta

5 followers · 0 following
View GitHub Profile
@drldcsta
drldcsta / gist:d809fcb25ae32aebf914
Last active February 13, 2019 06:55
using curl -w $curl_format to pretty print curl output
#/usr/bin/env bash
curl_format='
time_namelookup: %{time_namelookup}
time_connect: %{time_connect}
time_appconnect: %{time_appconnect}
time_pretransfer: %{time_pretransfer}
time_redirect: %{time_redirect}
time_starttransfer: %{time_starttransfer}
@drldcsta
drldcsta / gist:4fdeb5a6a49e4dee7849
Last active August 29, 2015 14:12
breaking down the rap monument youtube video into direct links to each rappers verse (in awk...of course)
@drldcsta
drldcsta / gist:891a53f8243a56574bb2
Created December 24, 2014 07:00
AirBoxOmega:~ d$ pbpaste |fgrep title=|grep -i duration|awk -F"[<>]" '{print $5}'|head -23
Ghosts - Sniper Wolf
Mekon feat &#39;Mad&#39; Frankie Fraser Revenge Of The Mekon
Junior Murvin - Police &amp; Thieves (HQ)
E-Z Rollers - Walk this Land
Oasis - Fuckin&#39; in the bushes (Snatch)
Birdy - 1901 [Official Music Video]
The Cure - Close To Me
The Chemical Brothers - Star Guitar
Shinichi Osawa - Star Guitar
@drldcsta
drldcsta / find.in.list.awk
Created August 26, 2014 21:27
Simple script to take a list of values from one file and search for it in another
BEGIN {
SUBSEP=" "
FS=","
while (getline < FILE )
{
desired_ids[$0]++
}
}
{
if ($3 in desired_ids) {
@drldcsta
drldcsta / words.awk
Created June 26, 2014 16:22
A co-worker (who will not be named) was bragging how quickly he could spin up an EMR cluster and write a query to get a word count for the works of Dickens. I bet him I could do it faster in awk. I won.
#!/usr/local/bin/gawk -f
BEGIN{max_word_count=0}
{
i = 1
while (i <= NF) {
word[$i]++
if (word[$i] > max_word_count) {
max_word_count=word[$i]
max_word=$i
}
@drldcsta
drldcsta / prettyTrace.awk
Created March 16, 2014 23:59
#!/bin/awk
#before processing lines, check if "src" variable has been passed
#If so, print it out, if not just print "origin"
#Set the field separator to be '(' or ')'
BEGIN {
if (src) {
printf "%s -> ", src
} else {
printf "%s -> ", "origin"
}
@drldcsta
drldcsta / streamBreakdown.msg.awk
Created March 12, 2014 23:43
#!/usr/bin/env awk
function calPer(n, d)
{
if (n == 0) {
return n
} else {
return substr(n / d * 100, 0, 5)
}
}
@drldcsta
drldcsta / gist:7777126
Created December 3, 2013 20:41
awk profiling
[root@mgmt01 logs]# time zfgrep -vw CM manager.log.7.gz|awk --profile 'BEGIN{print "Date,Time,FX SessID,USRM SessID,Home,Asset";OFS=","};$7 == 17018{split($NF,HA,"[$_]"); A[$8] = HA[3];H[$8] = HA[4]} $7 == 17031 && $0 ~ /available/ || $0 ~ /unusable/ {dec=substr($8,14);ses=substr($8,2,12);dec_ses[$8]=ses"/"strtonum("0x"dec); print $1,$2,$8,dec_ses[$8],H[$8],A[$8];home[H[$8]]++};END{print "home,# of errors";K = asorti(home,n);for (i=1;i<=K;i++) print n[i],home[n[i]]} '
Date,Time,FX SessID,USRM SessID,Home,Asset
12/02/13,18:44:16.636,@00223A25759201250256,00223A257592/19202646,35674934,65019706
12/02/13,19:13:21.379,@001AC3FDF72C04530C7A,001AC3FDF72C/72551546,36959215,64824121
12/02/13,19:38:54.079,@ECE09B07960105E4B090,ECE09B079601/98873488,43795953,65057265
12/02/13,19:47:34.777,@001AC3FDF72C04530C7E,001AC3FDF72C/72551550,36959215,64824121
home,# of errors
35674934,1
36959215,2
43795953,1
@drldcsta
drldcsta / gist:7378397
Created November 8, 2013 22:02
parsing threadpool log
[root@proxy01 tmp]# wc -l lix.tp
157317 lix.tp
[root@proxy01 tmp]# head -5 lix.tp
10/25/13 20:00.01 Info Public Queue Size for A_COMP is 0
10/25/13 20:00.01 Info Private Queue Size for A_COMP is 15061
10/25/13 20:00.01 Warning Combined Threadpool size for A_COMP has exceeded 10000
10/25/13 20:00.01 Info Public Queue Size for PRO-POOL is 0
10/25/13 20:00.01 Info Private Queue Size for PRO-POOL is 0
[root@proxy01 tmp]# time awk '{SUBSEP=" ";OFS=",";H=substr($2,0,2);T=$(NF-2)}
H > 19 && H < 24 && /Info/ && T == "HELLO" && $4 == "Public" {t[$1,$2]++;HPUB[$1,$2]=$NF}
@drldcsta
drldcsta / gist:6709729
Created September 26, 2013 03:59
#Notes in response to http://bit.ly/1fFMKIx (really cool blog)
output=$(fgrep -c CRON /var/log/messages) #grave accent notation is depreciated
##no reason for 2 pipes when 0 will do
#if you're not using regex, use fgrep
if [[ -n ${output} ]];then #we can avoid the `local` issue by just checking the existence
echo "Found ${output}..." #of output
#exit 0 #you can set your exit status explicitly, but not a big deal
else
echo "No instances of..."