Skip to content

Instantly share code, notes, and snippets.

@xuf12
Created April 7, 2018 16:06
Show Gist options
  • Star 3 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save xuf12/8d7ddd92ec143d448a1772c8a538a88d to your computer and use it in GitHub Desktop.
Save xuf12/8d7ddd92ec143d448a1772c8a538a88d to your computer and use it in GitHub Desktop.
loading text files to pandas
def get_texts(path):
rows_list = []
for idx, label in enumerate(CLASSES):
print(f'working on {path}/{label}')
for fname in (path/f'{label}').glob('*.*'):
dict1 = {}
text = fname.open('r').read()
dict1.update({
'text':text,
'label':idx
})
rows_list.append(dict1)
print(len(rows_list))
df = pd.DataFrame(rows_list)
return df
df = get_texts(PATH/'data_raw')
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment