Skip to content

Instantly share code, notes, and snippets.

@josiahcarlson
Created April 13, 2014 18:03
Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save josiahcarlson/10595010 to your computer and use it in GitHub Desktop.
Save josiahcarlson/10595010 to your computer and use it in GitHub Desktop.
Python function to offer "tail -F"-like functionality in Python
'''
follow_file.py
Written April 13, 2014 by Josiah Carlson
Released under the MIT license.
This could likely be made more efficient with the use of the pyinotify library,
and some of the module-level constants could be tuned for better performance,
depending on your platform.
'''
import os
import time
CHUNKSIZE = 2**20
ZERO_READ_CHECK = 10
ZERO_READ_SLEEP = .001 # in seconds
def file_follower(fname, callback, offset=0):
'''
This function will read from the file provided by `fname` until it has been
replaced. Each chunk of data that is read is passed to the provided
`callback(offset, chunk)`, where `offset` is the position in the file from
where the chunk of data was read from.
You can provide an optional starting `offset` if you would like to
continue reading from an arbitrary position.
'''
while 1:
with open(fname, 'rb') as inp:
if offset:
# pick up where you left off
inp.seek(offset)
no_data = 0
while 1:
# read a chunk
chunk = inp.read(CHUNKSIZE)
if chunk:
# send the chunk to the callback
callback(offset, chunk)
no_data = 0
offset += len(chunk)
continue
no_data += 1
if not no_data % ZERO_READ_CHECK:
# check for file replacement
if os.stat(fname).st_size < offset:
# signals that this file is done
callback(offset, "")
break
# maybe just a slow writer, pause for a bit
time.sleep(ZERO_READ_SLEEP)
# start from the beginning with the new file
offset = 0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment