Instantly share code, notes, and snippets.

Embed
What would you like to do?
Proof of concept: streaming a conference video while inlaying the slides thanks to SBCs

Streaming a conference video while inlaying the slides

So, we have a target solution with closed source software and DSLR Camera, but what if we found a cheaper and open-source solution with one or several SBCs? I asked the question on Twitter and on Armbian forum.

Table of contents

Table of contents generated with markdown-toc

Hardware

Existing

I have access to:

Possible

We should get

Magnets at Armbian forum said he used HDMI to Network converter. It looks promising!

Brian Linuxing suggested to use a NanoPC-T4 to record and encode, but it is based on a RK3399 SoC, and I already booked a Renegade Elite with the same SoC for September.

NanoPC-T4

It supports GPU and VPU acceleration:

  • GPU: Mali-T864 GPU,supports OpenGL ES1.1/2.0/3.0/3.1, OpenVG1.1, OpenCL, DX11, and AFBC
  • VPU: 4K VP9 and 4K 10bits H265/H264 60fps decoding, Dual VOP, etc

The interesting part is that the video input is one or two 4-Lane Mipi CSI, dual ISP ((Image Sensor Processor), up to 13MPix/s per ISP,supports simultaneous input of dual camera data...

Renegade Elite

As it is based on the same RK3399 SoC, it also has

Camera

Here is a quick comparison of CSI vs USB. It looks like CSI is way more powerful than USB because of bandwith, direct access to GPU, and other reasons. The problem is most of my boards don't have a CSI connector; the Raspberry Pi is the only one. 😳

The NanoPi-Fire3 has a DVP interface, but the camera choice seems to be smaller...

The Renegade Elite will have a CSI interface, but it will not be available before September, and it could take a few months to get it working under Armbian.

I will try to understand ISP/CSI/DVP and make a choice... I'm interested in the ArduCam cameras, but I'm not sure at all about the focal length. With the Omnivision OV5647 sensor, what focal length lens should I get?

Update 10/11/18

I finally bought an ArduCam camera, the 5MP one.

Sound

Jabra SPEAK 510

As we had one laying around, I chose to use a Jabra SPEAK 510 USB, and it seems to work quite easily with a Raspberry Pi. I had to modify /lib/modprobe.d/aliases.conf to comment options snd-usb-audio index=-2 and to create /etc/modprobe.d/alsa-base.conf:

# This sets the index value of the cards but doesn't reorder.
options snd_usb_audio index=0
options snd_bcm2835 index=1

# Does the reordering.
options snd slots=snd_usb_audio,snd_bcm2835

Then, the following commands produce the sound I was looking for:

 sudo arecord -vv -d 30 -fdat foo.wav; sudo aplay -D default foo.wav
External Mic/USB Sound card

I was lent a Sennheiser freeport and the output is XLR-3 or 6.35 Jack. Unfortunately, the Raspberry Pi doesn't take XLR, or even Jack as an audio input. You have to buy an audio HAT, an USB Audio interface or an USB sound card.

So... I bought myself an USB sound card and a cable to link the mic base to the sound card. This SSS1629 chip is supposed to work under Linux. This way, we could use whatever we have on hand (Orange Pi Zero?) and not be married for eternity to Raspberry Pi.

It looks like colleagues will get this kind of mics, so I will have to test soon with them.

HDMI Splitter/Extender

For the time being, I just got a LKV373A HDbitT HDMI 1080P Extender Reapter over RJ45 Cat5e/Cat6. The Splitter/Emitter was very tempting, but I haven't looked at the price yet.

Update 10/11/08

I have bought the splitter too.

A simple Splitter should do the trick (at about $35). I haven't been able to watch video coming from the TX on my Windows laptop, and the howto commands won't work for me, even on Linux VM. I tried this command under Linux on an ODroid XU4, and it somehow worked:

vlc --network-caching 16000 udp://@239.255.42.42:5004/

I now have a Check your TX Input or something approaching... Later on, I retried with

cvlc udp://@239.255.42.42:5004/

and I got the first image showing. Screenshot Unfortunately, I never get any other image of the remote computer, always the very first one is displayed. I also have errors:

odroid@gamestation-turbo:~$ cvlc udp://@239.255.42.42:5004/ 
VLC media player 2.2.8 Weatherwax (revision 2.2.7-14-g3cc1d8cba9)
[0004b268] dummy interface: using the dummy interface module...
Failed to open VDPAU backend libvdpau_nvidia.so: cannot open shared object file: No such file or directory
[abf04860] xcb_xv vout display error: no available XVideo adaptor
libGL error: MESA-LOADER: malformed or no PCI ID
libGL error: unable to load driver: exynos_dri.so
libGL error: driver pointer missing
libGL error: failed to load driver: exynos
[abd01418] ts demux error: libdvbpsi (PSI decoder): TS discontinuity (received 4, expected 7) for PID 17
[abd01418] ts demux error: libdvbpsi (PSI decoder): TS discontinuity (received 4, expected 7) for PID 0
[abd01418] ts demux error: libdvbpsi (PSI decoder): TS discontinuity (received 4, expected 7) for PID 4096
[abd0c628] avcodec decoder error: more than 5 seconds of late video -> dropping frame (computer too slow ?)

There is a software that should help display the videos. https://github.com/AdamLaurie/hdmi-rip

Software

Streaming just the video (and sound) of the speaker and recording the slides and the speaker video at the same time could be enough for the POC. We could rework the two videos afterwards to inlay the slides into the speaker video.

OBS

Fabio Loli and Ed Vielmetti proposed to use OBS as it is a well known product. Then Dennis S said it could not work on ARM without modification. That could be a job for WorksOnArm.

Voctomix

Dennis S then suggested Voctomix which happens to be available for ARM. I tried to install it on a Raspberry Pi I had in front of me, but the SDCard was already almost full, so it did not get to the end. I then installed it on on OrangePi Zero.

Nice talk about Voctomix: Voctomix Talk.

Yodeck

Yodeck, Raspberry Pi and HDMI TX Sender.

Current proposals

At the Raspberry Pi forum, I'm having an interesting conversation regarding other options. Why not using a good sound input card like the Audio Injector Ultra 2? To get the slides video, we could use a converter like the HDMI2CSI from Auvidea.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment