PythonFileOperations.md

Reading

Open file for reading

f = open("somefile.txt", "r")
print(f.read()) # print entire file
print(f.read(4)) # print first 4 characters of the file
print(f.readline()) # print first line 

Print file line by line

f = open("somefile.txt", "r")
for x in f:
  print(x)

Reading all files with extension

for file in glob.glob("*.txt"):

Reading files of different extensions

import glob
types = ('*.pdf', '*.cpp') # the tuple of file types
files_grabbed = []
for files in types:
    files_grabbed.extend(glob.glob(files))

files_grabbed   # the list of pdf and cpp files

Writing

Dataframe

csv file

df.to_csv(file_name)

tsv file

df.to_csv(file_name, sep='\t')

Lists

You can write list to a file one element at a time (see second option for large lists). NOTE _ = is a neat way to prevent f.write from printing out the number of characters written to stdout

with open("outfilename", 'w') as f:
  for item in mylist:
    _ = f.write(f"{item}\n")
    

For large lists, pickle is a good option, but only if you don't want any human reading the written list file (will be saved in a binary format)

# writing the list to a file
import pickle
l = [1,2,3,4]
with open("test.txt", "wb") as fp:   #Pickling
    pickle.dump(l, fp)

# reading the list 

with open("test.txt", "rb") as fp:   # Unpickling
    b = pickle.load(fp)
    

Write print statements directly to file

with open('out.txt', 'w') as f:
    print >> f, 'Filename:', filename     # Python 2.x
    print('Filename:', filename, file=f)  # Python 3.x
    

write downloaded file (from requests)

fcontent = response.text
file_out = open("filename","w")
file_out.write(fcontent)
file_out.close()

Checking

import os.path

os.path.isfile(path) # returns true if file exists

from pathlib import Path

my_file = Path("/path/to/file")
if my_file.is_file():
    # file exists

if my_file.is_dir():
    # directory exists

To check whether a Path object exists independently of whether is it a file or directory, use exists():

if my_file.exists():
    # path exists

Check whether the file is empty

import os

if os.stat(file).st_size == 0:
  print(file)
  

Removing

os.remove() #will remove a file.

os.rmdir() #will remove an empty directory.

shutil.rmtree() #will delete a directory and all its contents.