Skip to content

Instantly share code, notes, and snippets.

@lambdamusic

lambdamusic/split_json.sh

Created February 27, 2019 11:37
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save lambdamusic/50c8b92f8df8314b188b41c7fa4fc72b to your computer and use it in GitHub Desktop.
Save lambdamusic/50c8b92f8df8314b188b41c7fa4fc72b to your computer and use it in GitHub Desktop.
Download ZIP
Split jsonl file by line with 10000+ lines and rename files
Raw
split_json.sh
#!/bin/bash
FILENAME=records_111901_00000000.jsonl
split -a 5 -l 1 $FILENAME temp
n=1
for f in temp*
do
cat $f >> Part${n}.json
rm $f
((n++))
done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment