Skip to content

Instantly share code, notes, and snippets.

@yamingd
Created September 13, 2013 06:56
Show Gist options
  • Save yamingd/6547452 to your computer and use it in GitHub Desktop.
Save yamingd/6547452 to your computer and use it in GitHub Desktop.
split big file on first column's value
function splitBigFile()
{
tail -n 1000000000 $1 | awk '{
subNum=substr($1,length($1)-1)
print subNum,$0
}' | while read subNum others
do
echo $others >> $2.$subNum
done
}
@yamingd
Copy link
Author

yamingd commented Sep 13, 2013

tail -n 10000 member_pv.tmp | awk '{a=$1%100; print $0 > "mid_"a".txt"}'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment