Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
VTK Rendered to an FBO in a Qt Quick 2 Scene Graph

Integrating VTK into Qt Quick 2

This is a minimal proof-of-concept attempt at integrating VTK into a Qt Quick scene by rendering VTK to an OpenGL Framebuffer Object (FBO) and displaying that FBO in the QSG as a texture. This is a more robust integration approach than using the QSG beforeRendering or afterRendering signals (for example, as used by OpenView Core)

Build Instructions

Requires Qt 5.2 or greater. Tested configurations:

  • RHEL6, VTK 5.10, Qt 5.2.1
  • Ubuntu 12.04, VTK 6.1, Qt 5.3.0
  • Mac OS 10.9,VTK 6.1, Qt 5.2.1
git clone https://gist.github.com/c3fb01bb7ecaf437f7d6.git VtkFboInQtQuick
mkdir VtkFboInQtQuick-build
cd VtkFboInQtQuick-build

# may need to add: -DVTK_DIR:PATH=<VTK_DIR> where <VTK_DIR> is the
# path to the directory that contains VTKConfig.cmake
cmake ../VtkFboInQtQuick

make
./VtkFboInQtQuick
cmake_minimum_required(VERSION 2.8.11)
project(VtkFboInQtQuick)
set(CMAKE_INCLUDE_CURRENT_DIR ON)
set(CMAKE_AUTOMOC ON)
find_package(VTK REQUIRED)
include(${VTK_USE_FILE})
find_package(Qt5Quick REQUIRED)
configure_file(VtkFboInQtQuickConfig.h.in ${PROJECT_BINARY_DIR}/VtkFboInQtQuickConfig.h)
add_executable(${PROJECT_NAME}
main.cpp
QVTKFramebufferObjectItem.cpp
main.qml
)
target_link_libraries(${PROJECT_NAME} ${VTK_LIBRARIES} Qt5::Quick)
#include "VtkFboInQtQuickConfig.h"
#include "QVTKFramebufferObjectItem.h"
#include <vtkActor.h>
#include <vtkConeSource.h>
#include <vtkPolyDataMapper.h>
#include <vtkGenericOpenGLRenderWindow.h>
#include <vtkRenderer.h>
#include <vtkSmartPointer.h>
#include <vtkRendererCollection.h>
#include <vtkCamera.h>
#include <vtkProperty.h>
#include <QGuiApplication>
#include <QQuickView>
#include <QList>
int main(int argc, char **argv)
{
QGuiApplication app(argc, argv);
qmlRegisterType<QVTKFrameBufferObjectItem>("VtkQuick", 1, 0, "VtkRenderWindow");
QQuickView view;
view.setSource(QUrl(PROJECT_SOURCE_DIR "/main.qml"));
QList<QVTKFrameBufferObjectItem*> vtkItems = view.rootObject()->findChildren<QVTKFrameBufferObjectItem*>();
// For demonstration: Add a cone to the scene of each QVTKFrameBufferObjectItem
Q_FOREACH(QVTKFrameBufferObjectItem *vtkItem, vtkItems)
{
vtkGenericOpenGLRenderWindow *rw = vtkItem->GetRenderWindow();
// Create a renderer and add it to the render window
vtkSmartPointer<vtkRenderer> renderer = vtkSmartPointer<vtkRenderer>::New();
rw->AddRenderer(renderer);
vtkSmartPointer<vtkConeSource> polyDataSource = vtkSmartPointer<vtkConeSource>::New();
vtkSmartPointer<vtkPolyDataMapper> mapper = vtkSmartPointer<vtkPolyDataMapper>::New();
vtkSmartPointer<vtkActor> actor = vtkSmartPointer<vtkActor>::New();
mapper->SetInputConnection(polyDataSource->GetOutputPort());
actor->SetMapper(mapper);
actor->GetProperty()->SetOpacity(0.5); // demonstrate support for translucent VTK objects
renderer->AddActor(actor);
}
view.setResizeMode( QQuickView::SizeRootObjectToView );
view.show();
return app.exec();
}
import QtQuick 2.0
import VtkQuick 1.0
Rectangle {
id: root
width: 800
height: 600
color: "yellow"
Rectangle {
id : vtkRenderWindowContainer
anchors.fill: parent
anchors.margins: 20
border.width: 4
border.color: "black"
color: "blue"
VtkRenderWindow {
id: vtkRenderWindow
anchors.fill: parent
property real flipAngle: mouseArea.containsMouse ? 0 : 180
Behavior on flipAngle { NumberAnimation {} }
transform: Rotation {
origin { x: vtkRenderWindow.width/2; y: vtkRenderWindow.height/2 }
axis { x: 0; y: 1; z: 0 }
angle: vtkRenderWindow.flipAngle
}
}
}
Rectangle {
id: reactiveRect
color: "mintcream"
opacity: 0.8
radius: 10
border.width: 5
border.color: "mintcream"
width: 600
height: 200
anchors.centerIn: vtkRenderWindowContainer
rotation: mouseArea.containsMouse ? -4 : 4
Behavior on rotation { NumberAnimation {} }
Text {
id: label
anchors.fill: parent
anchors.margins: 20
color: mouseArea.containsMouse ? "deeppink" : "dodgerblue"
Behavior on color { ColorAnimation { duration: 200 } }
wrapMode: Text.WordWrap
font.pointSize: 16
font.bold: true
verticalAlignment: Text.AlignVCenter
text: "The objects inside the blue rectangle are rendered to an FBO by QVTKFrameBufferObject. Everything else is a standard Qt Quick component."
}
MouseArea {
id: mouseArea
anchors.fill: parent
hoverEnabled: true
}
}
}
#include "QVTKFramebufferObjectItem.h"
// Use the OpenGL API abstraction from Qt instead of from VTK because vtkgl.h
// and other Qt OpenGL-related headers do not play nice when included in the
// same compilation unit
#include <QOpenGLFunctions>
#include <QQuickFramebufferObject>
#include <QOpenGLFramebufferObject>
#include <vtkGenericOpenGLRenderWindow.h>
#include <vtkObjectFactory.h>
#include <vtkRendererCollection.h>
#include <vtkCamera.h>
class QVTKFramebufferObjectRenderer;
class vtkInternalOpenGLRenderWindow : public vtkGenericOpenGLRenderWindow, protected QOpenGLFunctions
{
public:
static vtkInternalOpenGLRenderWindow* New();
vtkTypeMacro(vtkInternalOpenGLRenderWindow, vtkGenericOpenGLRenderWindow)
virtual void OpenGLInitState()
{
Superclass::OpenGLInitState();
// Before any of the gl* functions in QOpenGLFunctions are called for a
// given OpenGL context, an initialization must be run within that context
this->MakeCurrent();
initializeOpenGLFunctions();
glUseProgram(0); // Shouldn't Superclass::OpenGLInitState() handle this?
glDisable(GL_DEPTH_TEST); // depth buffer fighting between the cone and the backround without this
glDisable(GL_BLEND); // doesn't seem crucial (?) but it is one of the differnces that showed up in apitrace analysis
}
// Override to use deferred rendering - Tell the QSG that we need to
// be rendered which will then, at the appropriate time, call
// InternalRender to do the actual OpenGL rendering.
virtual void Render();
// Do the actual OpenGL rendering
void InternalRender()
{
Superclass::Render();
}
// Provides a convenient way to set the protected FBO ivars from an existing
// FBO that was created and owned by Qt's FBO abstraction class
// QOpenGLFramebufferObject
void SetFramebufferObject(QOpenGLFramebufferObject *fbo);
QVTKFramebufferObjectRenderer *QtParentRenderer;
protected:
vtkInternalOpenGLRenderWindow() :
QtParentRenderer(0)
{
}
~vtkInternalOpenGLRenderWindow()
{
// Prevent superclass destructors from destroying the framebuffer object.
// QOpenGLFramebufferObject owns the FBO and manages it's lifecyle.
this->OffScreenRendering = 0;
}
};
vtkStandardNewMacro(vtkInternalOpenGLRenderWindow);
class QVTKFramebufferObjectRenderer : public QQuickFramebufferObject::Renderer
{
friend class vtkInternalOpenGLRenderWindow;
public:
QVTKFramebufferObjectRenderer(vtkInternalOpenGLRenderWindow *rw) :
m_vtkRenderWindow(rw),
m_framebufferObject(0)
{
m_vtkRenderWindow->Register(NULL);
m_vtkRenderWindow->QtParentRenderer = this;
}
~QVTKFramebufferObjectRenderer()
{
m_vtkRenderWindow->QtParentRenderer = 0;
m_vtkRenderWindow->Delete();
}
virtual void synchronize(QQuickFramebufferObject * item)
{
// the first synchronize call - right before the the framebufferObject
// is created for the first time
if (!m_framebufferObject)
{
QVTKFrameBufferObjectItem *vtkItem = static_cast<QVTKFrameBufferObjectItem*>(item);
vtkItem->init();
}
}
// Called from the render thread when the GUI thread is NOT blocked
virtual void render()
{
m_vtkRenderWindow->PushState();
m_vtkRenderWindow->OpenGLInitState();
m_vtkRenderWindow->InternalRender(); // vtkXOpenGLRenderWindow renders the scene to the FBO
m_vtkRenderWindow->PopState();
// Dolly camera back and forth - FOR DEMONSTRATION PURPOSES ONLY
static int callCount = 0;
++callCount;
double dolly = 1.0 + ((callCount % 200) > 100 ? -0.001 : 0.001);
m_vtkRenderWindow->GetRenderers()->GetFirstRenderer()->GetActiveCamera()->Dolly(dolly);
this->update();
}
QOpenGLFramebufferObject *createFramebufferObject(const QSize &size)
{
qDebug("QVTKFramebufferObjectRenderer::createFramebufferObject");
QOpenGLFramebufferObjectFormat format;
format.setAttachment(QOpenGLFramebufferObject::Depth);
m_framebufferObject = new QOpenGLFramebufferObject(size, format);
m_vtkRenderWindow->SetFramebufferObject(m_framebufferObject);
return m_framebufferObject;
}
vtkInternalOpenGLRenderWindow *m_vtkRenderWindow;
QOpenGLFramebufferObject *m_framebufferObject;
};
//
// vtkInternalOpenGLRenderWindow Definitions
//
void vtkInternalOpenGLRenderWindow::Render()
{
if (this->QtParentRenderer)
{
this->QtParentRenderer->update();
}
}
void vtkInternalOpenGLRenderWindow::SetFramebufferObject(QOpenGLFramebufferObject *fbo)
{
// QOpenGLFramebufferObject documentation states that "The color render
// buffer or texture will have the specified internal format, and will
// be bound to the GL_COLOR_ATTACHMENT0 attachment in the framebuffer
// object"
this->BackLeftBuffer = this->FrontLeftBuffer = this->BackBuffer = this->FrontBuffer =
static_cast<unsigned int>(GL_COLOR_ATTACHMENT0);
// Save GL objects by static casting to standard C types. GL* types
// are not allowed in VTK header files.
QSize fboSize = fbo->size();
this->Size[0] = fboSize.width();
this->Size[1] = fboSize.height();
this->NumberOfFrameBuffers = 1;
this->FrameBufferObject = static_cast<unsigned int>(fbo->handle());
this->DepthRenderBufferObject = 0; // static_cast<unsigned int>(depthRenderBufferObject);
this->TextureObjects[0] = static_cast<unsigned int>(fbo->texture());
this->OffScreenRendering = 1;
this->OffScreenUseFrameBuffer = 1;
this->Modified();
}
//
// QVTKFrameBufferObjectItem Definitions
//
QVTKFrameBufferObjectItem::QVTKFrameBufferObjectItem()
{
m_win = vtkInternalOpenGLRenderWindow::New();
}
QVTKFrameBufferObjectItem::~QVTKFrameBufferObjectItem()
{
m_win->Delete();
}
QQuickFramebufferObject::Renderer *QVTKFrameBufferObjectItem::createRenderer() const
{
return new QVTKFramebufferObjectRenderer(static_cast<vtkInternalOpenGLRenderWindow*>(m_win));
}
vtkGenericOpenGLRenderWindow *QVTKFrameBufferObjectItem::GetRenderWindow() const
{
return m_win;
}
void QVTKFrameBufferObjectItem::init()
{
qDebug("QVTKFrameBufferObjectItem::init");
}
#ifndef QVTKFrameBufferObjectItem_h_
#define QVTKFrameBufferObjectItem_h_
#include <QtQuick/QQuickFramebufferObject>
class vtkGenericOpenGLRenderWindow;
class QVTKFramebufferObjectRenderer;
class QVTKFrameBufferObjectItem : public QQuickFramebufferObject
{
Q_OBJECT
public:
QVTKFrameBufferObjectItem();
~QVTKFrameBufferObjectItem();
Renderer *createRenderer() const;
vtkGenericOpenGLRenderWindow* GetRenderWindow() const;
protected:
// Called once before the FBO is created for the first time. This method is
// called from render thread while the GUI thread is blocked.
virtual void init();
vtkGenericOpenGLRenderWindow *m_win;
friend class QVTKFramebufferObjectRenderer;
};
#endif
#ifndef VTKFBOINQTQUICKCONFIG_H_IN
#define VTKFBOINQTQUICKCONFIG_H_IN
#define PROJECT_SOURCE_DIR "@PROJECT_SOURCE_DIR@"
#define PROJECT_BINARY_DIR "@PROJECT_BINARY_DIR@"
#endif // VTKFBOINQTQUICKCONFIG_H_IN
@iwegner

This comment has been minimized.

Copy link

commented Nov 19, 2014

Nice prototype... to make it work on my system (win7 VC12 x64) I had to add ::fromLocalFile in main function ln 25:
view.setSource(QUrl::fromLocalFile(PROJECT_SOURCE_DIR "/main.qml"));

So works on my system as well with a separate render thread activated by setting
set QML_FORCE_THREADED_RENDERER="1" in a batch file that sets the local variables and starts the solution that is located in a directory called "build":

set QT5_DIR=C:/
set VTK_DIR=C:/
PATH=%QT5_DIR%/bin/;%VTK_DIR%/bin;%PATH%
set QML_FORCE_THREADED_RENDERER="1"
"build/VtkFboInQtQuick.sln"

Thank's for the code!

@elmewo

This comment has been minimized.

Copy link

commented Dec 8, 2014

Thanks for the prototype! Is there any chance to use the z-buffer? I played around with this, if I activate it there isn't even z fighting. It's simply not rendering the vtk scene.
As it looks now there is a culling problem with the cone in this solution.

@as2587

This comment has been minimized.

Copy link

commented Sep 18, 2015

Thanks!

Is there a way to set an interactor style to the render window and forward mouse events to the VTK layer?

@nbigaouette

This comment has been minimized.

Copy link

commented Nov 30, 2015

Unfortunately the final VTK shown on screen is flipped in Y. While you don't really see it using a cone, doing the same with a vtkChartXY shows the flipping.

I suspect it comes from VTK expecting (0,0) to be the top left corner while QML would be bottom-left?

Whatever the reason, how can the fbo be flipped in y?

@qaler

This comment has been minimized.

Copy link

commented Dec 14, 2015

Thank you nocnokneo for your greate effort in vtk merging to qml.
I am also curious about adding interactor style to qml VtkRenderWindow, could you kindly point a way to do it?

@tomkulaga

This comment has been minimized.

Copy link

commented Dec 29, 2015

Hi,
This is really nice thanks for the effort.
in main.cpp

line 25, i have to change:
view.setSource(QUrl(PROJECT_SOURCE_DIR "/main.qml"));
to
view.setSource(QUrl::fromLocalFile(PROJECT_SOURCE_DIR "//main.qml"));

to get it to work under Win10 on Qt 5.6. Otherwise i get a "network error" when the qml loads up.

@comaso

This comment has been minimized.

Copy link

commented Jan 8, 2016

Thank you so much!

For information, I tested it with Windows 10, Qt 5.6 beta and VTK 6.3 and it works too (with the change proposed by tomkulaga above to load the qml file). However, it crashes with VTK 7.0 RC when Superclass::OpenGLInitState() is being called in vtkInternalOpenGLRenderWindow. Since VTK 7 uses the new rendering engine (OpenGL2), I suspect it's because of that. The new rendering engine of VTK is supposed to be way faster so that would be really cool to make it work too. I'll take a look and report if I find anything.

Edit: it crashes with VTK 6.3 also if the option OpenGL2 renderer is selected in CMake. So there's definitely something wrong with the latest rendering engine of VTK (http://www.kitware.com/source/home/post/144).

@comaso

This comment has been minimized.

Copy link

commented Jan 8, 2016

I just found this from VTK: http://kitware.com/blog/home/post/688. It sounds like an API ideally designed for integrating with QtQuick... No?

@comaso

This comment has been minimized.

Copy link

commented Jan 13, 2016

For those interested, I managed to make it work. In the QVTKFrameBufferObjectItem class, I used a vtkExternalOpenGLRenderWindow instead of a vtkGenericOpenGLRenderWindow. I passed directly the vtkExternalOpenGLRenderWindow object to the QVTKFramebufferObjectRenderer's constructor. No need to define a vtkInternalOpenGLRenderWindow anymore. And in the render() method of QVTKFramebufferObjectRenderer I simply call Render() on this vtkExternalOpenGLRenderWindow object. It works with VTK 7.0 rc (and therefore the "new" OpenGL2 backend of VTK).

@iceroom17

This comment has been minimized.

Copy link

commented Jan 18, 2016

@comaso I am very interested in this topic.
I replaced vtkGenericOpenGLRenderWindow by vtkExternalOpenGLRenderWindow as you said. And I used
m_vtkRenderWindow->Render() in the render() method of QVTKFramebufferObjectRenderer as you said. But it crashed with VTK7.0rc.

I don't know how to handle the "m_vtkRenderWindow->SetFramebufferObject(m_framebufferObject)" in the method "QOpenGLFramebufferObject *createFramebufferObject(const QSize &size)"

Could you show your solution code more clearly? Thank you!

@comaso

This comment has been minimized.

Copy link

commented Jan 18, 2016

@iceroom17 Sure. As I said, I don't even have the class vtkInternalOpenGLRenderWindow anymore, so no SetFrameBufferObject. I'm not an expert in VTK nor OpenGL, I'm just a guy trying to make it work. I'm not saying everyone should do what I did, I might have been too naive, I don't know. But I can display my vtkImageActor inside a Qt interface and it does not crash.

In the example above, m_win is a pointer of type vtkGenericOpenGLRenderWindow. But in practice, QQuickFramebufferObject::Renderer *QVTKFrameBufferObjectItem::createRenderer() does a static_cast on the pointer and turns in into a pointer of vtkInternalOpenGLRenderWindow. In my implementation m_win is directly a vtkExternalOpenGLRenderWindow and stays a vtkExternalOpenGLRenderWindow pointer. I don't make use of vtkInternalOpenGLRenderWindow, which does not exist anymore. createRenderer() simply returns new QVTKFramebufferObjectRenderer(m_win). And for the class QVTKFramebufferObjectRenderer, I only implement render() with a call to "m_win->Render()". Nothing else.

It's awfully simple and I might be lucky in seeing it work. I'd be interested in understanding it better. In the header file of vtkExternalOpenGLRenderWindow is written the following:

"This class extends vtkGenericOpenGLRenderWindow to allow sharing the same OpenGL context by various visualization applications. Basically, this class prevents VTK from creating a new OpenGL context. Thus, it requires that an OpenGL context be initialized before Render is called."

I'm assuming that's why I don't need to call any OpenGLInitState(). Qt has already created a valid OpenGL context. So we can call Render() directly. vtkGenericOpenGLRenderWindow does not create a new window and simply uses the existing context to make its draw calls. I understand that the context chosen by Qt does not necessarily have the right options for VTK and VTK might change things in the context that may cause some issue with Qt. For this reason, it's probably the cleanest solution. But for me, it works. Is it clearer now? I'd be very interested in hearing if it worked for you.

@nocnokneo any input on the matter? You probably have a better understanding of all of this.

@comaso

This comment has been minimized.

Copy link

commented Jan 19, 2016

Just a quick note. In the render() method of QVTKFramebufferObjectRenderer, you should also call the method start(). So it should look something like this:
m_win->Start();
m_win->Render();

The call to the Start() method makes sure all important OpenGL options are set for VTK and also computes the size of the render window (without that, the app will crash as soon as you resize the application window).

I had an issue when I exported QSG_VISUALIZE=overdraw (see at the end of http://doc.qt.io/qt-5.5/qtquick-visualcanvas-scenegraph-renderer.html). When running the code with my NVIDIA 960M it would crash after a few seconds. If the code was running on the Intel 4600, it works ok. This troubled me for a while. Adding the call to the Start() method solved it. And looking at the implementation of it, it definitely should be called before rendering.

@robbie-li

This comment has been minimized.

Copy link

commented Jan 20, 2016

Hi comaso,
Now I am using vtk 7.0 rc2 and QT 5.5, Previously I managed to vtkGenericOpenGLRenderWindow but create a new opengl context.
Now followed your suggestion I am using the vtkExternalOpenGLRenderWindow.

I add a TrackballMouseInteracter for vtkRenderWindow, but now matter which renderwindow I use, I always faced the following issue:

  1. OpenGL error output in vtkOutputWindow for every mouse action.
    ERROR: In E:\GitRepo\vtk\Rendering\OpenGL2\vtkOpenGLRenderer.cxx, line 441
    vtkOpenGLRenderer (08C720F8): failed after Clear 16 OpenGL errors detected
    0 : (1282) Invalid operation
    1 : (1282) Invalid operation
    ......
  2. Crash after mouse rotation, mouse zoom in and out works well
    sometimes in vtkAbstractTransform::GetMaxMTime
    sometimes in some CriticalSection.
@comaso

This comment has been minimized.

Copy link

commented Jan 21, 2016

I don't think you can do that. I first attempted to add a vtkRenderWindowInteractor to my renderer too but it crashes fairly often. I was responding to Qt even (mouse, wheel, etc.) and calling methods from the interactor to forward the events. The issue is that the QuickItem is executed in the main thread so the events are received and processed on the main thread. But the interactor executes some OpenGL functions and there is no OpenGL context. It's probably something similar in your case.

For implementing a zoom in the image I displayed for instance, I implemented wheelEvent(QWheelEvent * p_event) in the QuickItem and managed the position of the VTK renderers'camera directly. I was implementing my own zoom via a custom vtkInteractorStyleImage so I was kind of doing that already anyway. And I no longer use any interactor.

@MichalGallovic

This comment has been minimized.

Copy link

commented Jan 25, 2016

Hey @nocnokneo, great gist!

I would like to do the same without QT, just using vTK.

What I wan to do, is just render vtkRenderWindow into the Frame buffer. Could it be done this way ?

vtkRenderWindow -> vtkXOpenGLRenderWindow -> vtkFrameBufferObject - at this point I can access FBO and use it ?

I think I know how to get to the point of having vtkXOpenGLRenderWindow, but not sure, how to make vtkFrameBufferObject of it... any help would be really appreciated, thanks !

@nocnokneo

This comment has been minimized.

Copy link
Owner Author

commented Apr 5, 2016

Hi everyone. Apologies for not responding sooner. I never got any comment notifications - not even for @mentions - due to GitHub bug #21. Since there is no sign that this bug has been fixed I doubt that any of you will get notifications for my comment here either. Frustrating.

I'm not actively working on this any longer, but I hope this work and all the derivative forks that I see can live on in the place where they belong: right next to QVTKWidget in the VTK code base. So to get that process started I've created VTK bug #16061.

In the meantime, you may soon find a more polished/productized version of this code in MITK thanks to the folks on the GraphMIC team.

@augspurger

This comment has been minimized.

Copy link

commented Apr 7, 2016

I've been trying to use this code in an existing qml GUI, but I've run into a problem with opacity: I can adjust the opacity of vtkActors in the window, but cannot get them to be truly opaque even when opacity is set to 1.0. This problem shows up, too, when I run the code above on its own, too--if you switch to a CylinderSource, for example, and set opacity to 1.0, you can still see both caps on the cyliner. Anybody have any clue how this might be fixed?

@zaferl

This comment has been minimized.

Copy link

commented Jan 26, 2018

Would really appreciate it if someone could post the VTK OpenGL2 compatible code, I am a newbie and could not make it work..

@nicanor-romerovenier

This comment has been minimized.

Copy link

commented Jul 11, 2018

I have created a repository with code that integrates VTK with Qt QuickControls 2, tested with VTK 8 and Qt 5.9. You can find the code here https://github.com/nicanor-romero/QtVtk with building instructions in the README. Also a brief article about it,
https://blog.bq.com/es/integrating-qtquickcontrols-2-with-vtk/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.