Skip to content

Instantly share code, notes, and snippets.

@changtimwu
Created January 17, 2024 02:16
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save changtimwu/bea5c9e8988915dfd071cbb1f07bb1a7 to your computer and use it in GitHub Desktop.
Save changtimwu/bea5c9e8988915dfd071cbb1f07bb1a7 to your computer and use it in GitHub Desktop.
frigate source code

go2rtc

work with go2rtc by API

  • in the Dockerfile, go2rtc is integrated in the form of executable instead of package/library

frontend

  • It's based on preact. No typescript.
  • No widget libs like MUI is used. They implemented their own.
iei@iei-POCm-ADL:~/frigate/web/src$ ls
api         app.css  assets      context  hooks  index.css  __mocks__    routes       __tests__  vite-env.d.ts
AppBar.jsx  app.tsx  components  env.js   icons  main.tsx   preact.d.ts  Sidebar.jsx  utils
  • the region editor is based canvas
iei@iei-POCm-ADL:~/frigate/web/src$ grep -i canvas  `find . -name "*.*sx"
`
./components/CameraImage.jsx:  const canvasRef = useRef(null);
./components/CameraImage.jsx:      if (canvasRef.current) {
./components/CameraImage.jsx:        const ctx = canvasRef.current.getContext('2d');
./components/CameraImage.jsx:    [img, scaledHeight, scaledWidth, setHasLoaded, onload, canvasRef]
./components/CameraImage.jsx:    if (!config || scaledHeight === 0 || !canvasRef.current) {
./components/CameraImage.jsx:  }, [apiHost, canvasRef, name, img, searchParams, scaledHeight, config]);
./components/CameraImage.jsx:        <canvas data-testid="cameraimage-canvas" height={scaledHeight} ref={canvasRef} width={scaledWidth} />
./components/JSMpegPlayer.jsx:      if (video.els.canvas.webkitRequestFullScreen) {
./components/JSMpegPlayer.jsx:        video.els.canvas.webkitRequestFullScreen();
./components/JSMpegPlayer.jsx:        video.els.canvas.mozRequestFullScreen();
./components/JSMpegPlayer.jsx:    video.els.canvas.addEventListener('click',fullscreen);
@changtimwu
Copy link
Author

changtimwu commented Jan 17, 2024

Back End

it's based on Flask and Blueprint

from flask import (
    Blueprint,
    Flask,
    Response,
    current_app,
    escape,
    jsonify,
    make_response,
    request,
)

@changtimwu
Copy link
Author

changtimwu commented Jan 17, 2024

IPC with queue
app.py

    def init_queues(self) -> None:
        # Queues for clip processing
        self.event_queue: Queue = mp.Queue()
        self.event_processed_queue: Queue = mp.Queue()
        self.video_output_queue: Queue = mp.Queue(
            maxsize=sum(camera.enabled for camera in self.config.cameras.values()) * 2
        )

        # Queue for cameras to push tracked objects to
        self.detected_frames_queue: Queue = mp.Queue(
            maxsize=sum(camera.enabled for camera in self.config.cameras.values()) * 2
        )

        # Queue for object recordings info

@changtimwu
Copy link
Author

changtimwu commented Jan 17, 2024

@changtimwu
Copy link
Author

changtimwu commented Jan 17, 2024

recent improvements on rk3588

@changtimwu
Copy link
Author

changtimwu commented Jan 17, 2024

@changtimwu
Copy link
Author

changtimwu commented Jan 17, 2024

@changtimwu
Copy link
Author

there are modules under thefrigate directory

detectors

events

comms

motion

ptz

record

track

test

util

@changtimwu
Copy link
Author

changtimwu commented Jan 17, 2024

inference hardware module
https://github.com/blakeblackshear/frigate/tree/dev/frigate/detectors/plugins

$ ls detectors/plugins/
cpu_tfl.py      deepstack.py    edgetpu_tfl.py  __init__.py     openvino.py     rknn.py         tensorrt.py

some highlights of plugin(aka hal)

  • the base class is DetectionApi
class OvDetector(DetectionApi):
    type_key = DETECTOR_KEY
def detect_raw(self, tensor_input):

call from object_detection.py

@changtimwu
Copy link
Author

decode with ffmpeg

@changtimwu
Copy link
Author

[+] Running 1/1
 ✔ Container frigate  Recreated                                                                               7.5s
 Attaching to frigate
 frigate  | s6-rc: info: service s6rc-fdholder: starting
 frigate  | s6-rc: info: service s6rc-oneshot-runner: starting
 frigate  | s6-rc: info: service s6rc-oneshot-runner successfully started
 frigate  | s6-rc: info: service fix-attrs: starting
 frigate  | s6-rc: info: service s6rc-fdholder successfully started
 frigate  | s6-rc: info: service fix-attrs successfully started
 frigate  | s6-rc: info: service legacy-cont-init: starting
 frigate  | s6-rc: info: service legacy-cont-init successfully started
 frigate  | s6-rc: info: service log-prepare: starting
 frigate  | s6-rc: info: service log-prepare successfully started
 frigate  | s6-rc: info: service nginx-log: starting
 frigate  | s6-rc: info: service go2rtc-log: starting
 frigate  | s6-rc: info: service frigate-log: starting
 frigate  | s6-rc: info: service go2rtc-log successfully started
 frigate  | s6-rc: info: service go2rtc: starting
 frigate  | s6-rc: info: service nginx-log successfully started
 frigate  | s6-rc: info: service frigate-log successfully started
 frigate  | s6-rc: info: service go2rtc successfully started
 frigate  | s6-rc: info: service go2rtc-healthcheck: starting
 frigate  | s6-rc: info: service frigate: starting
 frigate  | s6-rc: info: service go2rtc-healthcheck successfully started
 frigate  | s6-rc: info: service frigate successfully started
 frigate  | s6-rc: info: service nginx: starting
 frigate  | s6-rc: info: service nginx successfully started
 frigate  | s6-rc: info: service legacy-services: starting
 frigate  | 2024-01-17 18:32:34.789344854  [INFO] Preparing go2rtc config...
 frigate  | 2024-01-17 18:32:34.789442521  [INFO] Starting Frigate...
 frigate  | 2024-01-17 18:32:34.790659990  [INFO] Starting NGINX...
 frigate  | s6-rc: info: service legacy-services successfully started
 frigate  | 2024-01-17 18:32:35.072475417  [INFO] Starting go2rtc...
 frigate  | 2024-01-17 18:32:35.222640365  18:32:35.221 INF go2rtc version 1.2.0 linux/amd64
 frigate  | 2024-01-17 18:32:35.222647301  18:32:35.221 INF [api] listen addr=:1984
 frigate  | 2024-01-17 18:32:35.222651548  18:32:35.222 INF [rtsp] listen addr=:8554
 frigate  | 2024-01-17 18:32:35.222655060  18:32:35.222 INF [srtp] listen addr=:8443
 frigate  | 2024-01-17 18:32:35.222659007  18:32:35.222 INF [webrtc] listen addr=:8555
 frigate  | 2024-01-17 18:32:35.639410931  [2024-01-17 18:32:35] frigate.app                    INFO    : Starting Frigate (0.12.1-367d724)
 frigate  | 2024-01-17 18:32:35.651398255  [2024-01-17 18:32:35] frigate.config                 WARNING : Customizing more than a detector model path is unsupported.
 frigate  | 2024-01-17 18:32:35.654000103  [2024-01-17 18:32:35] peewee_migrate                 INFO    : Starting migrations
 frigate  | 2024-01-17 18:32:35.656953929  [2024-01-17 18:32:35] peewee_migrate                 INFO    : There is nothing to migrate
 frigate  | 2024-01-17 18:32:35.664481597  [2024-01-17 18:32:35] detector.ov                    INFO    : Starting detection process: 1013
 frigate  | 2024-01-17 18:32:35.667053686  [2024-01-17 18:32:35] frigate.app                    INFO    : Output process started: 1014
 frigate  | 2024-01-17 18:32:35.667207298  [2024-01-17 18:32:35] frigate.app                    INFO    : Camera processor started for topo1: 1018
 frigate  | 2024-01-17 18:32:35.669100499  [2024-01-17 18:32:35] frigate.app                    INFO    : Camera processor started for mytplink: 1020
 frigate  | 2024-01-17 18:32:35.671044255  [2024-01-17 18:32:35] frigate.app                    INFO    : Capture process started for topo1: 1021
 frigate  | 2024-01-17 18:32:35.673333881  [2024-01-17 18:32:35] frigate.app                    INFO    : Capture process started for mytplink: 1025
 frigate  | 2024-01-17 18:32:36.094049454  [2024-01-17 18:32:36] frigate.app                    WARNING : The current SHM size of 67.1MB is too small, recommend increasing it to at least 69.1MB.
 frigate  | 2024-01-17 18:32:36.109509131  [2024-01-17 18:32:36] frigate.detectors.plugins.openvino INFO    : Model Input Shape: {1, 300, 300, 3}
 frigate  | 2024-01-17 18:32:36.109562010  [2024-01-17 18:32:36] frigate.detectors.plugins.openvino INFO    : Model Output-0 Shape: {1, 1, 100, 7}
 frigate  | 2024-01-17 18:32:36.109594314  [2024-01-17 18:32:36] frigate.detectors.plugins.openvino INFO    : Model has 1 Output Tensors
 frigate  | 2024-01-17 18:32:44.789814585  [INFO] Starting go2rtc healthcheck service...


@changtimwu
Copy link
Author

changtimwu commented Jan 17, 2024

image
  • TOPO1
[{"return_code":0,"stderr":"","stdout":{"programs":[],"streams":[{"avg_frame_rate":"15/1","codec_long_name":"H.264/AVC/MPEG-4AVC/MPEG-4part10","height":720,"width":1280},{"avg_frame_rate":"0/0","bit_rate":"64000","codec_long_name":"PCMA-law/G.711A-law"}]}}]
  • TPLINK
[{"return_code":0,"stderr":"","stdout":{"programs":[],"streams":[{"avg_frame_rate":"15/1","codec_long_name":"H.264/AVC/MPEG-4AVC/MPEG-4part10","height":480,"width":640},{"avg_frame_rate":"0/0","bit_rate":"64000","codec_long_name":"PCMA-law/G.711A-law"}]}}]

@changtimwu
Copy link
Author

go2rtc has its own dashboard(web ui)
http:// /live/webrtc/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment