Skip to content

Instantly share code, notes, and snippets.

@cadrev
Forked from u8sand/downsample.py
Created July 31, 2017 08:55
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save cadrev/2ad73d5d1b84047faaac2db6dec938d0 to your computer and use it in GitHub Desktop.
Save cadrev/2ad73d5d1b84047faaac2db6dec938d0 to your computer and use it in GitHub Desktop.
Downsample lines in a file, useful for csv's that are too big.
#!/bin/python
'''
Usage:
python downsample.py [offset+]amount
Examples:
cat super_big.csv | python downsample.py 1+4 > big_divided_by_4.csv
cat data.csv | python downsample.py 2 > data_halved.csv
'''
import sys
from itertools import cycle
if len(sys.argv)==1:
print('Must specify downsample amount ([offset+]amount)', file=sys.stderr)
else:
arg = sys.argv[1]
if arg.find('+')!=-1:
delay, amnt = map(int, arg.split('+'))
for a in range(delay):
sys.stdout.write(next(sys.stdin))
else:
amnt = int(arg)
n = cycle(range(amnt))
for l in sys.stdin:
if next(n) == 0:
sys.stdout.write(l)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment