Qt and foreign windows

qt-foreign-windows.md

Qt and foreign windows

This document intends to establish an overview of the state of foreign window handling in Qt today, and how we should proceed to improve the support for foreign windows.

Terms and concepts

When describing windows in the following text we typically refer to a QWindow, unless prefixed with native, in which case we refer to the native backing window that the platforms's QPlatformWindow uses, represented by WId in the Qt APIs.

Each platform has a different internal representation of the native window, as outlined in the following table:

Platform	Native window (WId)	Other handles used
OS X (Cocoa)	NSView	NSWindow
iOS	UIView
Linux (xcb)	xcb_window_t
Windows	HWND
Android	jobject

Use-cases

There are two major use-cases for combining Qt windows with native windows.

Embedding Qt windows inside native windows

This is the use-case where you have a QWindow representing Qt content, e.g. a widgets hierarchy, a Qt Quick application, etc, and you want to embed the Qt content inside a native window, such as a host application that loads Qt as a plugin and presents one of its views.

This is the use-case covered by QAxServer in the Active Qt module, QMacNativeWidget in the Qt Widgets module, and the QX11EmbedWidget class in Qt 4.

Embedding native windows inside Qt windows

This is the use-case where you have a native window representing some content, either a single item such as a push button, or a hierarchy of items such as a web browser view, and you want to embed the native content inside a QWindow, possibly inside a child QWindow.

This is the use-case covered by QAxContainer in the Active Qt module, QMacCocoaViewContainer in the Qt Widgets module, and the QX11EmbedContainer class in Qt 4.

Qt APIs

QWindow

The QWindowclass has a few APIs directly related to interacting with native windows:

QWindow *QWindow::fromWinId(WId id)

Static factory method to create a QWindow wrapper around a native window handle. This sets the flag Qt::ForeignWindowon the resulting window.

WId QWindow::winId() const

Returns the window's native window handle. For regular windows created and controlled by Qt, this is the internal native window handle. For foreign windows this is the window handle that was passed to QWindow::fromWinId().

QPlatformWindow

The QPlatformWindow class has a matching winId function,

WId QPlatformWindow::winId() const

but doesn't have a factory function to match fromWinId(). Creating a new QPlatformWindow for a foreign QWindow is done by setting a magic property named _q_foreignWinId on the QWindow.

FIXME: We should add a proper API for this between QWindow and QPlatformWindow.

There's also a helper-API to determine if a given QPlatformWindow is a descendant of a foreign window:

bool QPlatformWindow::isEmbedded(const QPlatformThemeWindow *parentWindow) const

The default behaviour is to walk the parent hierarchy to the top, looking for a foreign window. If the parentWindow argument is non-null it will only check the given parent.

FIXME: QWindowsWindow::isEmbedded() will traverse the parent window hierarchy, while QXcbWindow::isEmbedded() will only check the immediate parent.

This API is used by QWindow::mapToGlobal() to determine if it needs to call QPlatformWindow::mapToGlobal(), or if it can use generic Qt code in QWindowPrivate::globalPosition() that just sums all the window positions in the window hierarchy.

FIXME: This logic should be baked into QWindowPrivate::globalPosition(), which should walk the parent hierarchy until it finds a foreign window, and then call QPlatformWindow::mapToGlobal().

It's also used in QApplicationPrivate::isWindowBlocked() to check if one window is (modally) blocked by another in the case where the two windows are no in the parent child chain.

FIXME: These windows should be in the parent child chain.

Finally, the API is used by QGuiApplication::topLevelWindows() to filter out foreign windows, which don't have a QWindow parent.

FIXME: This logic should actually check if the foreign window is a top level, and in that case include the foreign QWindow in the list of top level windows, not just exclude all foreign windows.

QPlatformNativeInterface

The QPlatformNativeInterface class provides an abstraction for retrieving native resource handles, including window resources:

void *QPlatformNativeInterface::nativeResourceForWindow(const QByteArray &, QWindow *)

Most platforms allow retrieving the same handle as returned by QWindow::winId(), e.g. by passing "nsview" as the resource on OS X, "handle" on Window, or "uiview" on iOS. Other internal window handles used by the platform may also be available, such as as NSWindow on OS X — retrieved by passing "nswindow".

Other

There's a global Qt::WindowType enum value ForeignWindow, used internally by QWindow when creating foreign windows in QWindow::fromWinId().

QPlatformIntegration has a Capability enum value ForeignWindow, used to check if the platform supports foreign windows at all.QWindow::fromWinId() will return 0 if the platform doesn't support foreign windows.

Supporting the two use-cases with Qt

There are many ways to serve the two use-cases with todays Qt APIs, some better than others.

Embedding Qt windows inside native windows

The most basic way, which has existed since Qt 4 times, is using QWindow::winId() to pull out the native handle for a QWindow, and then re-parent the native handle into a native window:

nativeWindow->addChild(qtWindow->winId()); // Pull out native handle and re-parent

A similar approach, but using QPlatformNativeInterface would be:

nativeWindow->addChild(nativeInterface->nativeResourceForWindow("nativehandle", qtWindow)); 

Both these approaches are less than ideal, as the embedding results in the native handle being re-parented behind Qt's back, causing the QWindow window hierarchy to get out of sync with the native window hierarchy. In theory the platform can detect the re-parenting and react by e.g. setting the QWindow parent of the embedded window to 0, but it still means that the rest of Qt thinks the QWindow is a top level window.

A slightly safer approach is using QWindow::fromWinId():

qtWindow->setParent(QWindow::fromWinId(nativeWindow)); // Re-parent using Qt APIs

This creates a QWindow wrapper around the native window that we want to re-parent into. — the wrapper being a foreign window with its type set to ForeignWindow.

With this approach Qt knows about the embedding, and can make informed decisions when traversing the window parent chain and hitting a ForeignWindow, such as mapping to global position using native APIs.

The last option is to use one of the platform-specific classes, QAxServer, QMacNativeWidget, or QX11EmbedWidget, which in most cases just wraps one of the above approaches.

FIXME: Deprecate the platform-specific classes in favour of QWindow::fromWinId() if we can.

Embedding native windows inside Qt windows

Just like for the opposite use-case, the most basic way is to use QWindow::winId() to get the native handle for a QWindow, and then use native APIs to add the native window as a child of the QWindow:

qtWindow->winId()->addChild(nativeWindow); // Add native window as child using native APIs

The same approach, but using QPlatformNativeInterface would be:

nativeInterface->nativeResourceForWindow("nativehandle", qtWindow)->addChild(nativeWindow);

Again, like for the opposite use-case, this leaves Qt completely out of the loop as to what the relationship between the two native windows are. When iterating children of the QWindow, we'll miss out on the native window, as it's not part of QObject::children().

The safer approach is using QWindow::fromWinId():

QWindow::fromWinId(nativeWindow)->setParent(qtWindow); // Add as child using Qt APIs

This creates a QWindow wrapper around the native window that we want to add as a child — the wrapper being a foreign window with its type set to ForeignWindow.

With this approach Qt knows about the embedding, and can make informed decisions when looking at QObject::children() and hitting a ForeignWindow.

The last option is to use one of the platform-specific classes, QAxContainer, QMacCocoaViewContainer , or QX11EmbedContainer , which in most cases just wraps one of the above approaches.

FIXME: Deprecate the platform-specific classes in favour of QWindow::fromWinId() if we can.

Issues with the current Qt APIs

The two uses-cases seem to been solved current Qt APIs, but there are quite a few traps in these APIs, and it's easy to shoot yourself in the foot or end up with undefined or platform-specific behaviour. There are also a number of open bugs about QWindow::fromWinId(), QWindow::winId(), and foreign windows in general.

One source of these issues is that QWindow::winId() and QPlatformNativeInterface::nativeResourceForWindow() leave Qt out of the loop.

FIXME: Document that QWindow::winId() and QPlatformNativeInterface::nativeResourceForWindow() should only be used when the native APIs used to manipulate the window can be guaranteed to not result in re-parenting of the native window. If re-parenting is a possibility, or done explicitly, QWindow::fromWinId() should be used.

For the platforms where it's possible we should try to detect when re-parenting happens, and output warnings about the situation, in addition to syncing the window hierarchy as much as we can. This may include using QWindow::fromWinId() under the hood to create a wrapper QWindow for the new native parent window.

A larger issue with our current APIs, that hasn't been discussed yet, is the fact that QWindow::fromWinId() returns a QWindow pointer, which from an API contract point of view should support any operation that any other QWindow supports, including using setters to manipulate the window, and connecting to signals to observe changes to the window.

This contract is not adhered to in practice by any of our platforms, and the documentation for QWindow::fromWinId() doesn't mention anything about the situation.

FIXME: Document that using the resulting QWindow * from QWindow::fromWinId() for any sort of manipulation or event observing is likely to fail and/or have platform specific behaviours.

The reasons for this undefined/platform specific behaviour largely boils down to our platforms relying on having full control of the native window handle, and the native window handle often being a subclass of the native window handle type, where we implement callbacks and other logic. When replacing the native window handle with an instance we don't control, and which doesn't implement our callback logic, the behaviour becomes undefined and full of holes compared to a regular QWindow.

FIXME: We should create a cross-platform implementation of QPlatformWindow for foreign windows that limits the API surface that is passed on to the real platform window. All non-supported API usages should output warnings and be no-ops in terms of behaviour.

Steps towards better support for foreign windows

1. Document QWindow::fromWinId() as being fragile/platform specific
2. Document QWindow::winId() as not supporting re-parenting
3. Limit API surface of QWindow returned by QWindow::fromWinId()
4. Deprecate platform-specific classes in favour of QWindow::fromWinId() where possible, or implement them in terms of QWindow::fromWinId() instead of QWindow::winId()
5. Evaluate which parts of the QWindow API surface can be supported cross platform and which parts can be supported for a subset of the platforms, and whether or not we want to invest time in adding this support.
6. Add support for the subset of the QWindow API surface that makes sense