Skip to content

Instantly share code, notes, and snippets.

@mhanney
Last active April 18, 2021 08:46
Show Gist options
  • Save mhanney/dc329b1d9bff032b3600a1f7b44099e5 to your computer and use it in GitHub Desktop.
Save mhanney/dc329b1d9bff032b3600a1f7b44099e5 to your computer and use it in GitHub Desktop.
Stream webcam and audio source from PC to remote IP address using mpegts encapsulation and encoded with h264 and aac

Requirement

Stream webcam and audio source from PC via the internet to a vmix install with as little lag as possible.

This could by transport stream UDP or TCP as long as the encapsulation is mpegts and encoded with h264 and aac.

Solution ideally needs to be windows based

Solution - Windows

Using ffmpeg-20161130-4e6d1c1-win64-static.zip for 64 bit Windows, Windows 10, which includes libx264

https://ffmpeg.zeranoe.com/builds/win64/static/

ffmpeg version N-82713-g4e6d1c1 Copyright (c) 2000-2016 the FFmpeg developers
  built with gcc 5.4.0 (GCC)
  configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-dxva2 --enable-libmfx --enable-nvenc --enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libschroedinger --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-lzma --enable-decklink --enable-zlib
  libavutil      55. 41.101 / 55. 41.101
  libavcodec     57. 66.108 / 57. 66.108
  libavformat    57. 58.101 / 57. 58.101
  libavdevice    57.  2.100 / 57.  2.100
  libavfilter     6. 67.100 /  6. 67.100
  libswscale      4.  3.101 /  4.  3.101
  libswresample   2.  4.100 /  2.  4.100
  libpostproc    54.  2.100 / 54.  2.100
Hyper fast Audio and Video encoder
usage: ffmpeg [options] [[infile options] -i infile]... {[outfile options] outfile}...

start-stream.bat

ffmpeg -loglevel error ^
-f dshow ^
-i video="USB Video Device":audio="Microphone (USB Audio Device)" ^
-vcodec libx264 ^
-preset ultrafast ^
-tune zerolatency ^
-thread_type slice ^
-slices 1 ^
-intra-refresh 1 ^
-r 30 ^
-g 60 ^
-s 800x600 ^
-aspect 4:3 ^
-acodec aac ^
-ar 44100 ^
-b:v 2.5M ^
-minrate:v 900k ^
-maxrate:v 2.5M ^
-bufsize:v 5M ^
-b:a 128K ^
-pix_fmt yuv420p ^
-f mpegts udp://192.168.0.123:35001?pkt_size=1316

Test with VLC

udp://@192.168.0.123:35001

Test with vMIX

Transport Stream over UDP
Port 35001

How to list available devices (for -i video="") on Windows

ffmpeg -list_devices true -f dshow -i dummy

Solution Mac OSX

ffmpeg installed using brew package manager on Yosemite

ffmpeg -loglevel error \
-f avfoundation \
-i "default" \
-vcodec libx264  \
-preset ultrafast \
-tune zerolatency \
-thread_type slice \
-slices 1 \
-intra-refresh 1 \
-r 30 \
-g 60 \
-s 800x600 \
-aspect 4:3 \
-acodec aac \
-ar 44100 \
-b:v 2.5M \
-minrate:v 900k \
-maxrate:v 2.5M \
-bufsize:v 5M \
-b:a 128K \
-pix_fmt yuv420p \
-f mpegts udp://192.168.0.123:35001?pkt_size=1316

How to list available devices (for -i video="") on Mac OSX

ffmpeg -f avfoundation -list_devices true -i ""

Example output (Late 2013 iMac 21.5")

[AVFoundation input device @ 0x7f89b9c22920] AVFoundation video devices:
[AVFoundation input device @ 0x7f89b9c22920] [0] FaceTime HD Camera (Built-in)
[AVFoundation input device @ 0x7f89b9c22920] [1] Capture screen 0
[AVFoundation input device @ 0x7f89b9c22920] AVFoundation audio devices:
[AVFoundation input device @ 0x7f89b9c22920] [0] Built-in Microphone

What all the arguments mean:

ffmpeg -loglevel error - logging level (quiet, error, warning).

-f dshow - use direct show as file type.

-i video="USB Video Device":audio="Microphone (USB Audio Device)" - inputs (use ffmpeg -list_devices true -f dshow -i dummy to get list of possible sources).

-vcodec libx264 - video codec (libx264 for h264).

-acodec aac - audio codec (aac).

-preset ultrafast - encoding preset (ultrafast, superfast, veryfast, faster, fast, medium, slow, slower, veryslow, placebo).

-tune zerolatency - tune zerolatency sends an I-Frame (complete pic) every frame so that users to not need to wait for intermetiate frames to complete.

-thread_type slice - slice-based threading tells all CPU threads work on the same frame, reducing latency a lot.

-slices 1 - has to be 1 for vMix to decode the stream, don't know why.

-intra-refresh 1 intra-refresh has to be set to 1 because vMix expects a latency of 1 frame, I think.

-r 30 - framerate.

-g 60 - GOP (Group of Pictures), simply multiply your output frame rate * 2.

-s 800x600 - scale my webcam's native picture is 1600x1200, so I scale it down.

-aspect 4:3 - aspect ratio, my webcam is a Logitec 9000 which is 4:3.

-acodec aac - audio encode using AAC.

-ar 44100 - audio sample rate 44.1 KHz.

-b:v 2.5M - desired bitrate for video 2.5 Mbps, can play with this.

-minrate:v 900k - min data rate video 900k, can play with this.

-maxrate:v 2.5M - min data rate video 2.5 Mbps, can play with this.

-bufsize:v 5M - buffer size for encoding, double max data rate seems to be a good starting point.

-b:a 128K - desired bitrate for audio 128 Kbps, can play with this.

-pix_fmt yuv420p - color space, has to be yuv420p for vMix to decode properly.

-bufsize 5000k - buffer size (double max data rate seems to be a good starting point)

-f mpegts udp://192.168.0.123:35001?pkt_size=1316 output format mpegts udp IP address, port, packet size - really important - The size of an MPEG-TS packet is 188 Bytes and 7 of them will fit into an Ethernet frame, 7*188=1316.

References

https://trac.ffmpeg.org/wiki/EncodingForStreamingSites

https://trac.ffmpeg.org/wiki/StreamingGuide

https://sites.google.com/site/linuxencoding/x264-ffmpeg-mapping

http://stackoverflow.com/questions/33624016/why-sliced-thread-affect-so-much-on-realtime-encoding-using-ffmpeg-x264

http://www.waveguide.se/?article=creating-dvb-t-compatible-mpeg2-streams-using-ffmpeg

http://stackoverflow.com/questions/12020430/look-for-fastest-video-encoder-with-least-lag-to-stream-webcam-streaming-to-ipad/12085571

ffmpeg -loglevel error ^
-f dshow ^
-i video="USB Video Device":audio="Microphone (USB Audio Device)" ^
-vcodec libx264 ^
-preset veryfast ^
-tune zerolatency ^
-thread_type slice ^
-slices 4 ^
-intra-refresh 1 ^
-r 30 ^
-s 800:600 ^
-aspect 4:3 ^
-acodec aac ^
-ar 44100 ^
-crf 22 ^
-b:a 128K ^
-maxrate:v 900k ^
-bufsize:v 90k ^
-pix_fmt yuv420p ^
-refs 1 ^
-subq 3 ^
-trellis 2 ^
-async 1 ^
-f mpegts udp://192.168.0.123:35001?pkt_size=1316
ffmpeg -loglevel error ^
-f dshow ^
-i video="USB Video Device":audio="Microphone (USB Audio Device)" ^
-vcodec libx264 ^
-preset ultrafast ^
-tune zerolatency ^
-thread_type slice ^
-slices 1 ^
-intra-refresh 1 ^
-r 30 ^
-g 60 ^
-s 800x600 ^
-aspect 4:3 ^
-acodec aac ^
-ar 44100 ^
-b:v 2.5M ^
-minrate:v 900k ^
-maxrate:v 2.5M ^
-bufsize:v 5M ^
-b:a 128K ^
-pix_fmt yuv420p ^
-f mpegts udp://192.168.0.123:35001?pkt_size=1316
@Ajinkz
Copy link

Ajinkz commented Mar 29, 2019

OS: windows 7
ffmpeg version N-93482-g9dece050ef
USB camera : Logictech C170

I tried running start-stream.bat file
On vlc console I got this

status change: ( new input: udp://localhost:35001 )
status change: ( play state: 3 )

But there was no video playing in vlc player

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment