Last active
January 18, 2020 22:15
-
-
Save dstandish/b1f4a5a93f5baba474bf74f00ca49d82 to your computer and use it in GitHub Desktop.
get specific lines from large compressed file with nonstandard row separator
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| a=$'\001' | |
| b=$'\002' | |
| c=$'\003' | |
| d=$'\004' | |
| ft="$a$b$c$c" # odd field separator | |
| rt="$a$b$c$d" # odd row separator | |
| zcat very_large_file.csv.gz | awk -v rt=$rt 'BEGIN { RS = rt } NR>=946241736&&NR<=946241740' > out.txt |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment