Streaming a conference video while inlaying the slides
So, we have a target solution with closed source software and DSLR Camera, but what if we found a cheaper and open-source solution with one or several SBCs? I asked the question on Twitter and on Armbian forum.
Table of contents
- Current proposals
I have access to:
- Two Orange Pi Zero
- Orange Pi One Plus
- Two Raspberry Pi 3B+
- Two Raspberry Pi 3B
- Odroid XU4
- NanoPi Fire3
- Renegade Elite in September
- video converter
We should get
It supports GPU and VPU acceleration:
- GPU: Mali-T864 GPU，supports OpenGL ES1.1/2.0/3.0/3.1, OpenVG1.1, OpenCL, DX11, and AFBC
- VPU: 4K VP9 and 4K 10bits H265/H264 60fps decoding, Dual VOP, etc
The interesting part is that the video input is one or two 4-Lane Mipi CSI, dual ISP ((Image Sensor Processor), up to 13MPix/s per ISP，supports simultaneous input of dual camera data...
As it is based on the same RK3399 SoC, it also has
Here is a quick comparison of CSI vs USB.
It looks like CSI is way more powerful than USB because of bandwith, direct access to GPU, and other reasons.
The problem is most of my boards don't have a CSI connector; the Raspberry Pi is the only one.
The NanoPi-Fire3 has a DVP interface, but the camera choice seems to be smaller...
The Renegade Elite will have a CSI interface, but it will not be available before September, and it could take a few months to get it working under Armbian.
I will try to understand ISP/CSI/DVP and make a choice... I'm interested in the ArduCam cameras, but I'm not sure at all about the focal length. With the Omnivision OV5647 sensor, what focal length lens should I get?
Jabra SPEAK 510
As we had one laying around, I chose to use a Jabra SPEAK 510 USB, and it seems to work quite easily with a Raspberry Pi.
I had to modify
/lib/modprobe.d/aliases.conf to comment
options snd-usb-audio index=-2 and to create
# This sets the index value of the cards but doesn't reorder. options snd_usb_audio index=0 options snd_bcm2835 index=1 # Does the reordering. options snd slots=snd_usb_audio,snd_bcm2835
Then, the following commands produce the sound I was looking for:
sudo arecord -vv -d 30 -fdat foo.wav; sudo aplay -D default foo.wav
External Mic/USB Sound card
I was lent a Sennheiser freeport and the output is XLR-3 or 6.35 Jack. Unfortunately, the Raspberry Pi doesn't take XLR, or even Jack as an audio input. You have to buy an audio HAT, an USB Audio interface or an USB sound card.
So... I bought myself an USB sound card and a cable to link the mic base to the sound card. This SSS1629 chip is supposed to work under Linux. This way, we could use whatever we have on hand (Orange Pi Zero?) and not be married for eternity to Raspberry Pi.
It looks like colleagues will get this kind of mics, so I will have to test soon with them.
I have bought the splitter too.
A simple Splitter should do the trick (at about $35). I haven't been able to watch video coming from the TX on my Windows laptop, and the howto commands won't work for me, even on Linux VM. I tried this command under Linux on an ODroid XU4, and it somehow worked:
vlc --network-caching 16000 udp://@220.127.116.11:5004/
I now have a Check your TX Input or something approaching... Later on, I retried with
odroid@gamestation-turbo:~$ cvlc udp://@18.104.22.168:5004/ VLC media player 2.2.8 Weatherwax (revision 2.2.7-14-g3cc1d8cba9) [0004b268] dummy interface: using the dummy interface module... Failed to open VDPAU backend libvdpau_nvidia.so: cannot open shared object file: No such file or directory [abf04860] xcb_xv vout display error: no available XVideo adaptor libGL error: MESA-LOADER: malformed or no PCI ID libGL error: unable to load driver: exynos_dri.so libGL error: driver pointer missing libGL error: failed to load driver: exynos [abd01418] ts demux error: libdvbpsi (PSI decoder): TS discontinuity (received 4, expected 7) for PID 17 [abd01418] ts demux error: libdvbpsi (PSI decoder): TS discontinuity (received 4, expected 7) for PID 0 [abd01418] ts demux error: libdvbpsi (PSI decoder): TS discontinuity (received 4, expected 7) for PID 4096 [abd0c628] avcodec decoder error: more than 5 seconds of late video -> dropping frame (computer too slow ?)
Streaming just the video (and sound) of the speaker and recording the slides and the speaker video at the same time could be enough for the POC. We could rework the two videos afterwards to inlay the slides into the speaker video.
Dennis S then suggested Voctomix which happens to be available for ARM. I tried to install it on a Raspberry Pi I had in front of me, but the SDCard was already almost full, so it did not get to the end. I then installed it on on OrangePi Zero.
Nice talk about Voctomix: Voctomix Talk.
Yodeck, Raspberry Pi and HDMI TX Sender.
At the Raspberry Pi forum, I'm having an interesting conversation regarding other options. Why not using a good sound input card like the Audio Injector Ultra 2? To get the slides video, we could use a converter like the HDMI2CSI from Auvidea.