Skip to content

Instantly share code, notes, and snippets.

@StrangeTcy
Last active July 12, 2021 11:24
Show Gist options
  • Save StrangeTcy/95cef1f596636694e5fed74a99e5c162 to your computer and use it in GitHub Desktop.
Save StrangeTcy/95cef1f596636694e5fed74a99e5c162 to your computer and use it in GitHub Desktop.
text_splitter
#!/usr/bin/bash
for f in *.txt_shard; do
mv -- "$f" "${f%.txt_shard}.txt"
FILENAME=${f%.txt_shard}.txt
FILESIZE=$(stat -c%s "$FILENAME")
echo "Size of $FILENAME = $FILESIZE bytes."
split --verbose --bytes 1MB $FILENAME $FILENAME --additional-suffix ".txt"
echo "Deleting $FILENAME"
rm -fv $FILENAME
done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment