Skip to content

Instantly share code, notes, and snippets.

@kbfreder
Last active April 20, 2021 18:15
Show Gist options
  • Save kbfreder/7999655da25f2e52fbbbd89fd360ce75 to your computer and use it in GitHub Desktop.
Save kbfreder/7999655da25f2e52fbbbd89fd360ce75 to your computer and use it in GitHub Desktop.
executing shell commands in python
# ref: https://janakiev.com/blog/python-shell-commands/
# 1. using os
import os
output = os.system('ls -l')
print(output)
# 2. using subprocess
import subprocess
# this example writes a dictionary (dict_obj) to a text file
put = subprocess.Popen(["hdfs", "dfs", "-put", "-", file_path],
stdin=subprocess.PIPE, text=True)
put.stdin.write(json.dumps(dict_obj))
put.stdin.close()
put.wait()
# this example captures output using 'cat'
file_current = subprocess.check_output(["hadoop", "fs", "-cat", file_path])
list_obj = filter(None, file_current.split(sep)) # split into array
# leverage an existing bash script
# first, make sure bash script is executable
d = 'path/to/dir'
subprocess.call(f'./erase_files_in_dir.sh {d} png', shell=True)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment