Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
#
# add to bashrc
#
# files
alias sampletree='mkdir -p sample/{train,test,valid}'
lsn(){ matchdir=`pwd`/$2; find $matchdir -type f | grep -v sample | shuf -n $1 | awk -F`pwd` '{print "."$NF}' ; }
# shuffle mv/cp
cpn(){ matchdir=`pwd`/$2; find $matchdir -type f | grep -v sample | shuf -n $1 | awk -F`pwd` '{print "."$NF" sample"$NF}' | xargs -t -n2 cp ; }
mvn(){ matchdir=`pwd`/$2; todir=`pwd`/$3; find $matchdir -type f | grep -v sample | shuf -n $1 | awk -F`pwd` -v todir="$todir" '{print $0" "todir}' | xargs -t -n2 mv ; }
# head mv/cp
cpnh(){ matchdir=`pwd`/$2; find $matchdir -type f | grep -v sample | head -n $1 | awk -F`pwd` '{print "."$NF" sample"$NF}' | xargs -t -n2 cp ; }
mvnh(){ matchdir=`pwd`/$2; todir=`pwd`/$3; find $matchdir -type f | grep -v sample | head -n $1 | awk -F`pwd` -v todir="$todir" '{print $0" "todir}' | xargs -t -n2 mv ; }
# tail mv/cp
cpnt(){ matchdir=`pwd`/$2; find $matchdir -type f | grep -v sample | tail -n $1 | awk -F`pwd` '{print "."$NF" sample"$NF}' | xargs -t -n2 cp ; }
mvnt(){ matchdir=`pwd`/$2; todir=`pwd`/$3; find $matchdir -type f | grep -v sample | tail -n $1 | awk -F`pwd` -v todir="$todir" '{print $0" "todir}' | xargs -t -n2 mv ; }
@brookisme
Copy link
Author

brookisme commented Mar 28, 2017

dogs and cats example

Assuming you are in the data directory with your test/valid/train directories already set up:

$ sampletree
$ mkdir sample/train/{dogs,cats} sample/valid/{dogs,cats}
$ cpn 200 train/dogs
$ cpn 200 train/cats
$ cpn 200 valid/dogs
$ cpn 200 valid/cats
$ cpn 500 test 

Similarly you can use the mvn command to set up your original training and validation sets.

mvn 2500 train/dogs valid/dogs
mvn 2500 train/cats valid/cats

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment