Skip to content

Instantly share code, notes, and snippets.

@thelinuxkid
Last active December 4, 2023 06:43
Show Gist options
  • Save thelinuxkid/5114777 to your computer and use it in GitHub Desktop.
Save thelinuxkid/5114777 to your computer and use it in GitHub Desktop.
Get a Python subprocess' output without buffering. Normally when you want to get the output of a subprocess in Python you have to wait until the process finishes. This is bad for long running processes. Here's a way to get the output unbuffered (in real-time.)
import contextlib
import subprocess
# Unix, Windows and old Macintosh end-of-line
newlines = ['\n', '\r\n', '\r']
def unbuffered(proc, stream='stdout'):
stream = getattr(proc, stream)
with contextlib.closing(stream):
while True:
out = []
last = stream.read(1)
# Don't loop forever
if last == '' and proc.poll() is not None:
break
while last not in newlines:
# Don't loop forever
if last == '' and proc.poll() is not None:
break
out.append(last)
last = stream.read(1)
out = ''.join(out)
yield out
def example():
cmd = ['ls', '-l', '/']
proc = subprocess.Popen(
cmd,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
# Make all end-of-lines '\n'
universal_newlines=True,
)
for line in unbuffered(proc):
print line
example()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment