Skip to content

Instantly share code, notes, and snippets.

@Vnicius
Created February 26, 2019 14:26
Show Gist options
  • Save Vnicius/bedae03a02358163632026419f6d92c7 to your computer and use it in GitHub Desktop.
Save Vnicius/bedae03a02358163632026419f6d92c7 to your computer and use it in GitHub Desktop.
import sys
import os
in_file = sys.argv[1]
try:
os.makedirs('source')
except:
pass
with open(in_file, 'r') as f:
lines = f.readlines()
for l in range(int(len(lines)/1000)):
path = os.path.join('source', f'wiki__{l}.txt')
with open(path, 'w') as out:
last = l * 1000 + 1000
last = last if last < len(lines) else len(lines)
for f in lines[l*1000 : last]:
out.write(f)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment