Skip to content

Instantly share code, notes, and snippets.

@locklin
Created September 12, 2014 11:36
Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save locklin/0f92fd032d4fa5233a1d to your computer and use it in GitHub Desktop.
Save locklin/0f92fd032d4fa5233a1d to your computer and use it in GitHub Desktop.
convert a 2-d csv to t7 format
#!/bin/bash
#
# Converts to torch7's serialized format
#
if [ $# -ne 2 ]
then
echo "Usage: $0 in out"
echo " Where in is the input csv and out is the torch file"
echo " The input csv must be a headerless 2d csv with the right number of carriage retuens"
echo " To read into torch, use the following:"
echo " file=torch.DiskFile(out,'r')"
echo " csvdat=file:readObject()"
exit 1
fi
echo "counting lines"
lines=$(wc -l < $1)
echo "lines = `echo $lines`"
cols=$(head -1 $1 | sed 's/[^,]//g' -| wc -c )
echo "columns = `echo $cols`"
nvals=$(($lines*$cols))
echo "creating header"
echo -e '4\n1\n3\nV 1\n18\ntorch.DoubleTensor\n2' > $2
echo "$lines $cols" >> $2
echo "$cols 1" >>$2
echo -e '1\n4\n2\n3\nV 1\n19\ntorch.DoubleStorage' >> $2
echo $nvals >> $2
echo "dumping data; this may take a moment..."
`cat $1 | tr -s ',\n' ' ' >> $2`
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment