Frame accurate video reader - OpenCV VideoCapture replacement

readme.md

OpenCV's VideoCapture is broken and hasn't been fixed for the last 5 years: opencv/opencv#9053

This is a PyAV based replacement. Unlike other implementations it can seek at any time.

How to use:

reader = VideoReader('video.mp4')
reader.seek(reader.total_frames - 100)  # frame number 
while True:
  frame = reader.read()
  if not frame:
    break
    
  # frame is an ndarray - do something with it
  print(f'frame {reader.position}: {frame}')
  
reader.close()

videoreader.py

import av


class VideoReader:
    def __init__(self, path):
        self.container = av.container.open(path)
        self.position = 0
        self.stream = self.container.streams.video[0]
        self.total_frames = self.stream.frames
        self.seek(0)

    def iter_frames(self):
        for packet in self.container.demux(self.stream):
            if packet.dts is None:
                continue
            for frame in packet.decode():
                yield frame

    def close(self):
        self.container.close()

    def read(self):
        try:
            frame = next(self.iter)
        except StopIteration:
            self.end = True
            return None
        self.position += 1
        return frame.to_rgb().to_ndarray(format='bgr24')

    def seek(self, frame):
        pts = int(frame * self.stream.duration / self.stream.frames)
        self.container.seek(pts, stream=self.stream)
        for j, f in enumerate(self.iter_frames()):
            if j > 100:
                raise RuntimeError('Did not find target within 100 frames of seek')
            if f.pts >= pts - 1:
                break
        self.end = False
        self.position = frame
        self.iter = iter(self.iter_frames())

Thanks for this @Eugeny and also @whizmo for your modifications. I've used this to make a version that just allows you to request frames directly without thinking about the current state of the reader. E.g.

reader = VideoReader(path="path/to/video.mp4')
frame = reader.request_frame(20)  # Ask for the 20th frame
cv2.imshow('frame', frame.image)
cv2.waitKey(1)

It should be fast when frames are requested in sequence, or when the requested frame was requested recently

Gist: https://gist.github.com/petered/db8e334c7aefdf367af1b11e6eefe733