Streaming the screen [as a camera] under Wayland

README.md

Preparation

Before beginning, you will need:

• v4l2loopback, a Linux kernel module that, like aloop, creates a loopback device that can be seen as a camera;
• gstreamer, which can integrate with pipewire but doesn't seem to use v4l2loopback devices well;
• ffmpeg, which does perfectly well with v4l2loopback but (for now) lacks pipewire support.

I'm presuming you're using pipewire and any of the xdg-desktop-portal implementations appropriate to your compositor, and that it is configured to be running as part of your session (and reachable via the DBus session bus).

Things to Note

The scripts copied below are fairly straightforward, but there are a few gotchas:

• exclusive_caps=1 ensures that the V4L2 "Loopback" camera looks like "a regular camera" after an application (here ffmpeg) starts using it as a destination. This is not strictly necessary for all applications, including Electron applications, but--if not done--may cause some other applications to dismiss the device as "not a camera".
• The node ID (it must be a pipewire Node) is extracted from the pipewire command line tools. This is technically a workaround to a bug in xdg-desktop-portal-wlr where, although the portal opened the pipewire node, it returned UINT32_MAX - 1 as the node ID--which also prevented screen capture from working in, among other things, Firefox.
• The color space is converted to yuyv422. The desktop portal usually streams in your native framebuffer colorspace, often rgba, but Electron/Chromium does not recognize devices without a "more typical" camera colorspace as being a camera.

Attribution

xdp-screen-cast.py is copied, with light modifications, from this GNOME snippet or a previous version (which is how I know the Node ID was wrong).

Starting the Stream

Run ./stream_screen.sh. It should print out helpful error messages, including the command line recommended for loading v4l2loopback:

	sudo modprobe v4l2loopback exclusive_caps=1 card_label=Loopback

... as well as when it can't find an application already having created a streaming node (in which case it recommends you run the included xdp-screen-cast.py).

If all goes well, you'll see ffmpeg count the frames being sent to the loopback device. At that point, it should exist as a camera, for all intents and purposes.

stream_screen.sh

#!/bin/bash

if ! lsmod | grep v4l2loopback > /dev/null; then
	cat >&2 <<EOF
Couldn't find v4l2loopback module. Try modprobing it? (Hint: it's usually good to include exclusive_caps=1, as such:)

	sudo modprobe v4l2loopback exclusive_caps=1 card_label=Loopback
EOF
	exit 1
fi

xdgnode="$(pw-cli ls Node | grep xdg-desktop -B 5 | grep -Eo "id ([[:digit:]]+)" | sed -e 's/id //g')"
if [ -z "$xdgnode" ]; then
	cat >&2 <<EOF
Couldn't find the xdg-desktop-portal PipeWire node. Currently, this can't be
started from a script because it requires passing an FD over DBus. (Write the
GNOME developers about this!)

In lieu of this, start an application (like Firefox) which can handle
screensharing, or, alternatively, try ~/Util/xdp-screen-cast.py .
EOF
	exit 1
fi
echo "Using PIPEWIRE_NODE=$xdgnode"

ents=( $(ls /dev/video*) )
if [ -n "$1" ]; then
	snk="$1"
else
	snk="${ents[-1]}"
	echo Assuming the sink V4L2 node is "$snk" "(if it's not, pass it as "'$1'")" >2
fi
echo "Using sink=$snk"

# FIXME: adjust screen res
# FIXME: -pixel_format used to be bgra
gst-launch -q pipewiresrc path=$xdgnode ! videoconvert ! filesink location=/dev/stdout | ffmpeg -f rawvideo -video_size 1366x768 -pixel_format rgba -framerate 30 -i /dev/stdin -f v4l2 -vf format=yuyv422 "$snk"

xdp-screen-cast.py

import time
import re
import signal
import dbus
from gi.repository import GLib
from dbus.mainloop.glib import DBusGMainLoop

import gi
gi.require_version('Gst', '1.0')
from gi.repository import GObject, Gst

GObject.threads_init()
DBusGMainLoop(set_as_default=True)
Gst.init(None)

loop = GLib.MainLoop()

bus = dbus.SessionBus()
request_iface = 'org.freedesktop.portal.Request'
screen_cast_iface = 'org.freedesktop.portal.ScreenCast'

pipeline = None

def terminate():
    if pipeline is not None:
        self.player.set_state(Gst.State.NULL)
    loop.quit()

request_token_counter = 0
session_token_counter = 0
sender_name = re.sub(r'\.', r'_', bus.get_unique_name()[1:])

def new_request_path():
    global request_token_counter
    request_token_counter = request_token_counter + 1
    token = 'u%d'%request_token_counter
    path = '/org/freedesktop/portal/desktop/request/%s/%s'%(sender_name, token)
    return (path, token)

def new_session_path():
    global session_token_counter
    session_token_counter = session_token_counter + 1
    token = 'u%d'%session_token_counter
    path = '/org/freedesktop/portal/desktop/session/%s/%s'%(sender_name, token)
    return (path, token)

def screen_cast_call(method, callback, *args, options={}):
    (request_path, request_token) = new_request_path()
    bus.add_signal_receiver(callback,
                            'Response',
                            request_iface,
                            'org.freedesktop.portal.Desktop',
                            request_path)
    options['handle_token'] = request_token
    method(*(args + (options, )),
           dbus_interface=screen_cast_iface)

def on_gst_message(bus, message):
    type = message.type
    if type == Gst.MessageType.EOS or type == Gst.MessageType.ERROR:
        terminate()

def play_pipewire_stream(node_id):
    empty_dict = dbus.Dictionary(signature="sv")
    fd_object = portal.OpenPipeWireRemote(session, empty_dict,
                                          dbus_interface=screen_cast_iface)
    fd = fd_object.take()
    print('starting stream with fd', fd, 'and node id', node_id)
    print('gonna hang out here and pretend useful work is happening now')
    #while True:
    #    time.sleep(3600)
    pipeline = Gst.parse_launch('pipewiresrc fd=%d path=%u ! videoconvert ! gtksink'%(fd, node_id))
    pipeline.set_state(Gst.State.PLAYING)
    pipeline.get_bus().connect('message', on_gst_message)

def on_start_response(response, results):
    if response != 0:
        print("Failed to start: %s"%response)
        terminate()
        return

    print('results:', results)
    print("streams:")
    for (node_id, stream_properties) in results['streams']:
        print("stream {}".format(node_id))
        play_pipewire_stream(node_id)

def on_select_sources_response(response, results):
    if response != 0:
        print("Failed to select sources: %d"%response)
        terminate()
        return

    print("sources selected")
    global session
    screen_cast_call(portal.Start, on_start_response,
                     session, '')

def on_create_session_response(response, results):
    if response != 0:
        print("Failed to create session: %d"%response)
        terminate()
        return

    global session
    session = results['session_handle']
    print("session %s created"%session)

    screen_cast_call(portal.SelectSources, on_select_sources_response,
                     session,
                     options={ 'multiple': False,
                               'types': dbus.UInt32(1|2) })

portal = bus.get_object('org.freedesktop.portal.Desktop',
                             '/org/freedesktop/portal/desktop')

(session_path, session_token) = new_session_path()
screen_cast_call(portal.CreateSession, on_create_session_response,
                 options={ 'session_handle_token': session_token })

try:
    loop.run()
except KeyboardInterrupt:
    terminate()