VTK Rendered to an FBO in a Qt Quick 2 Scene Graph

README.md

Integrating VTK into Qt Quick 2

This is a minimal proof-of-concept attempt at integrating VTK into a Qt Quick scene by rendering VTK to an OpenGL Framebuffer Object (FBO) and displaying that FBO in the QSG as a texture. This is a more robust integration approach than using the QSG beforeRendering or afterRendering signals (for example, as used by OpenView Core)

Build Instructions

Requires Qt 5.2 or greater. Tested configurations:

• RHEL6, VTK 5.10, Qt 5.2.1
• Ubuntu 12.04, VTK 6.1, Qt 5.3.0
• Mac OS 10.9,VTK 6.1, Qt 5.2.1

git clone https://gist.github.com/c3fb01bb7ecaf437f7d6.git VtkFboInQtQuick
mkdir VtkFboInQtQuick-build
cd VtkFboInQtQuick-build

# may need to add: -DVTK_DIR:PATH=<VTK_DIR> where <VTK_DIR> is the
# path to the directory that contains VTKConfig.cmake
cmake ../VtkFboInQtQuick

make
./VtkFboInQtQuick

CMakeLists.txt

cmake_minimum_required(VERSION 2.8.11)

project(VtkFboInQtQuick)

set(CMAKE_INCLUDE_CURRENT_DIR ON)
set(CMAKE_AUTOMOC ON)

find_package(VTK REQUIRED)
include(${VTK_USE_FILE})

find_package(Qt5Quick REQUIRED)

configure_file(VtkFboInQtQuickConfig.h.in ${PROJECT_BINARY_DIR}/VtkFboInQtQuickConfig.h)

add_executable(${PROJECT_NAME} 
  main.cpp
  QVTKFramebufferObjectItem.cpp
  main.qml
)
target_link_libraries(${PROJECT_NAME} ${VTK_LIBRARIES} Qt5::Quick)

main.cpp

#include "VtkFboInQtQuickConfig.h"
#include "QVTKFramebufferObjectItem.h"

#include <vtkActor.h>
#include <vtkConeSource.h>
#include <vtkPolyDataMapper.h>
#include <vtkGenericOpenGLRenderWindow.h>
#include <vtkRenderer.h>
#include <vtkSmartPointer.h>
#include <vtkRendererCollection.h>
#include <vtkCamera.h>
#include <vtkProperty.h>

#include <QGuiApplication>
#include <QQuickView>
#include <QList>

int main(int argc, char **argv)
{
   QGuiApplication app(argc, argv);

   qmlRegisterType<QVTKFrameBufferObjectItem>("VtkQuick", 1, 0, "VtkRenderWindow");

   QQuickView view;
   view.setSource(QUrl(PROJECT_SOURCE_DIR "/main.qml"));
   QList<QVTKFrameBufferObjectItem*> vtkItems = view.rootObject()->findChildren<QVTKFrameBufferObjectItem*>();

   // For demonstration: Add a cone to the scene of each QVTKFrameBufferObjectItem
   Q_FOREACH(QVTKFrameBufferObjectItem *vtkItem, vtkItems)
   {
      vtkGenericOpenGLRenderWindow *rw = vtkItem->GetRenderWindow();
      // Create a renderer and add it to the render window
      vtkSmartPointer<vtkRenderer> renderer = vtkSmartPointer<vtkRenderer>::New();
      rw->AddRenderer(renderer);

      vtkSmartPointer<vtkConeSource> polyDataSource = vtkSmartPointer<vtkConeSource>::New();
      vtkSmartPointer<vtkPolyDataMapper> mapper = vtkSmartPointer<vtkPolyDataMapper>::New();
      vtkSmartPointer<vtkActor> actor = vtkSmartPointer<vtkActor>::New();
      mapper->SetInputConnection(polyDataSource->GetOutputPort());
      actor->SetMapper(mapper);
      actor->GetProperty()->SetOpacity(0.5); // demonstrate support for translucent VTK objects
      renderer->AddActor(actor);
   }

   view.setResizeMode( QQuickView::SizeRootObjectToView );
   view.show();

   return app.exec();
}

main.qml

import QtQuick 2.0
import VtkQuick 1.0

Rectangle {
    id: root

    width: 800
    height: 600
    color: "yellow"

    Rectangle {
        id : vtkRenderWindowContainer
        anchors.fill: parent
        anchors.margins: 20
        border.width: 4
        border.color: "black"
        color: "blue"

        VtkRenderWindow {
            id: vtkRenderWindow
            anchors.fill: parent
            property real flipAngle: mouseArea.containsMouse ? 0 : 180
            Behavior on flipAngle { NumberAnimation {} }
            transform: Rotation {
                origin { x: vtkRenderWindow.width/2; y: vtkRenderWindow.height/2 }
                axis { x: 0; y: 1; z: 0 }
                angle: vtkRenderWindow.flipAngle
            }
        }
    }

    Rectangle {
        id: reactiveRect
        color: "mintcream"
        opacity: 0.8
        radius: 10
        border.width: 5
        border.color: "mintcream"

        width: 600
        height: 200

        anchors.centerIn: vtkRenderWindowContainer

        rotation: mouseArea.containsMouse ? -4 : 4
        Behavior on rotation { NumberAnimation {} }

        Text {
            id: label
            anchors.fill: parent
            anchors.margins: 20
            color: mouseArea.containsMouse ? "deeppink" : "dodgerblue"
            Behavior on color { ColorAnimation { duration: 200 } }
            wrapMode: Text.WordWrap
            font.pointSize: 16
            font.bold: true
            verticalAlignment: Text.AlignVCenter
            text: "The objects inside the blue rectangle are rendered to an FBO by QVTKFrameBufferObject. Everything else is a standard Qt Quick component."
        }

        MouseArea {
            id: mouseArea
            anchors.fill: parent
            hoverEnabled: true
        }
    }
}

QVTKFramebufferObjectItem.cpp

#include "QVTKFramebufferObjectItem.h"

// Use the OpenGL API abstraction from Qt instead of from VTK because vtkgl.h
// and other Qt OpenGL-related headers do not play nice when included in the
// same compilation unit
#include <QOpenGLFunctions>

#include <QQuickFramebufferObject>
#include <QOpenGLFramebufferObject>

#include <vtkGenericOpenGLRenderWindow.h>
#include <vtkObjectFactory.h>

#include <vtkRendererCollection.h>
#include <vtkCamera.h>

class QVTKFramebufferObjectRenderer;

class vtkInternalOpenGLRenderWindow : public vtkGenericOpenGLRenderWindow, protected QOpenGLFunctions
{
public:
  static vtkInternalOpenGLRenderWindow* New();
  vtkTypeMacro(vtkInternalOpenGLRenderWindow, vtkGenericOpenGLRenderWindow)

  virtual void OpenGLInitState()
  {
     Superclass::OpenGLInitState();

     // Before any of the gl* functions in QOpenGLFunctions are called for a
     // given OpenGL context, an initialization must be run within that context
     this->MakeCurrent();
     initializeOpenGLFunctions();

     glUseProgram(0); // Shouldn't Superclass::OpenGLInitState() handle this?
     glDisable(GL_DEPTH_TEST); // depth buffer fighting between the cone and the backround without this
     glDisable(GL_BLEND);      // doesn't seem crucial (?) but it is one of the differnces that showed up in apitrace analysis
  }

  // Override to use deferred rendering - Tell the QSG that we need to
  // be rendered which will then, at the appropriate time, call
  // InternalRender to do the actual OpenGL rendering.
  virtual void Render();

  // Do the actual OpenGL rendering
  void InternalRender()
  {
     Superclass::Render();
  }

  // Provides a convenient way to set the protected FBO ivars from an existing
  // FBO that was created and owned by Qt's FBO abstraction class
  // QOpenGLFramebufferObject
  void SetFramebufferObject(QOpenGLFramebufferObject *fbo);

  QVTKFramebufferObjectRenderer *QtParentRenderer;

protected:
  vtkInternalOpenGLRenderWindow() :
     QtParentRenderer(0)
  {
  }

  ~vtkInternalOpenGLRenderWindow()
  {
     // Prevent superclass destructors from destroying the framebuffer object.
     // QOpenGLFramebufferObject owns the FBO and manages it's lifecyle.
     this->OffScreenRendering = 0;
  }
};
vtkStandardNewMacro(vtkInternalOpenGLRenderWindow);


class QVTKFramebufferObjectRenderer : public QQuickFramebufferObject::Renderer
{
   friend class vtkInternalOpenGLRenderWindow;

public:
   QVTKFramebufferObjectRenderer(vtkInternalOpenGLRenderWindow *rw) :
      m_vtkRenderWindow(rw),
      m_framebufferObject(0)
   {
      m_vtkRenderWindow->Register(NULL);
      m_vtkRenderWindow->QtParentRenderer = this;
   }

   ~QVTKFramebufferObjectRenderer()
   {
      m_vtkRenderWindow->QtParentRenderer = 0;
      m_vtkRenderWindow->Delete();
   }

   virtual void synchronize(QQuickFramebufferObject * item)
   {
      // the first synchronize call - right before the the framebufferObject
      // is created for the first time
      if (!m_framebufferObject)
      {
         QVTKFrameBufferObjectItem *vtkItem = static_cast<QVTKFrameBufferObjectItem*>(item);
         vtkItem->init();
      }
   }

   // Called from the render thread when the GUI thread is NOT blocked
   virtual void render()
   {
      m_vtkRenderWindow->PushState();
      m_vtkRenderWindow->OpenGLInitState();
      m_vtkRenderWindow->InternalRender(); // vtkXOpenGLRenderWindow renders the scene to the FBO
      m_vtkRenderWindow->PopState();

      // Dolly camera back and forth - FOR DEMONSTRATION PURPOSES ONLY
      static int callCount = 0;
      ++callCount;
      double dolly = 1.0 + ((callCount % 200) > 100 ? -0.001 : 0.001);
      m_vtkRenderWindow->GetRenderers()->GetFirstRenderer()->GetActiveCamera()->Dolly(dolly);
      this->update();
   }

   QOpenGLFramebufferObject *createFramebufferObject(const QSize &size)
   {
      qDebug("QVTKFramebufferObjectRenderer::createFramebufferObject");
      QOpenGLFramebufferObjectFormat format;
      format.setAttachment(QOpenGLFramebufferObject::Depth);
      m_framebufferObject = new QOpenGLFramebufferObject(size, format);

      m_vtkRenderWindow->SetFramebufferObject(m_framebufferObject);

      return m_framebufferObject;
   }

   vtkInternalOpenGLRenderWindow *m_vtkRenderWindow;
   QOpenGLFramebufferObject *m_framebufferObject;
};

//
// vtkInternalOpenGLRenderWindow Definitions
//

void vtkInternalOpenGLRenderWindow::Render()
{
   if (this->QtParentRenderer)
   {
      this->QtParentRenderer->update();
   }
}

void vtkInternalOpenGLRenderWindow::SetFramebufferObject(QOpenGLFramebufferObject *fbo)
{
   // QOpenGLFramebufferObject documentation states that "The color render
   // buffer or texture will have the specified internal format, and will
   // be bound to the GL_COLOR_ATTACHMENT0 attachment in the framebuffer
   // object"
   this->BackLeftBuffer = this->FrontLeftBuffer = this->BackBuffer = this->FrontBuffer =
         static_cast<unsigned int>(GL_COLOR_ATTACHMENT0);

   // Save GL objects by static casting to standard C types. GL* types
   // are not allowed in VTK header files.
   QSize fboSize = fbo->size();
   this->Size[0] = fboSize.width();
   this->Size[1] = fboSize.height();
   this->NumberOfFrameBuffers = 1;
   this->FrameBufferObject       = static_cast<unsigned int>(fbo->handle());
   this->DepthRenderBufferObject = 0; // static_cast<unsigned int>(depthRenderBufferObject);
   this->TextureObjects[0]       = static_cast<unsigned int>(fbo->texture());
   this->OffScreenRendering = 1;
   this->OffScreenUseFrameBuffer = 1;
   this->Modified();
}

//
// QVTKFrameBufferObjectItem Definitions
//

QVTKFrameBufferObjectItem::QVTKFrameBufferObjectItem()
{
   m_win = vtkInternalOpenGLRenderWindow::New();
}

QVTKFrameBufferObjectItem::~QVTKFrameBufferObjectItem()
{
   m_win->Delete();
}

QQuickFramebufferObject::Renderer *QVTKFrameBufferObjectItem::createRenderer() const
{
   return new QVTKFramebufferObjectRenderer(static_cast<vtkInternalOpenGLRenderWindow*>(m_win));
}

vtkGenericOpenGLRenderWindow *QVTKFrameBufferObjectItem::GetRenderWindow() const
{
   return m_win;
}

void QVTKFrameBufferObjectItem::init()
{
   qDebug("QVTKFrameBufferObjectItem::init");
}

QVTKFramebufferObjectItem.h

#ifndef QVTKFrameBufferObjectItem_h_
#define QVTKFrameBufferObjectItem_h_

#include <QtQuick/QQuickFramebufferObject>

class vtkGenericOpenGLRenderWindow;
class QVTKFramebufferObjectRenderer;

class QVTKFrameBufferObjectItem : public QQuickFramebufferObject
{
   Q_OBJECT

public:
   QVTKFrameBufferObjectItem();
   ~QVTKFrameBufferObjectItem();
   Renderer *createRenderer() const;
   vtkGenericOpenGLRenderWindow* GetRenderWindow() const;

protected:
   // Called once before the FBO is created for the first time. This method is
   // called from render thread while the GUI thread is blocked.
   virtual void init();

   vtkGenericOpenGLRenderWindow *m_win;

   friend class QVTKFramebufferObjectRenderer;
};

#endif

VtkFboInQtQuickConfig.h.in

#ifndef VTKFBOINQTQUICKCONFIG_H_IN
#define VTKFBOINQTQUICKCONFIG_H_IN

#define PROJECT_SOURCE_DIR "@PROJECT_SOURCE_DIR@"
#define PROJECT_BINARY_DIR "@PROJECT_BINARY_DIR@"

#endif // VTKFBOINQTQUICKCONFIG_H_IN

I know this is an old thread, but I found that a small change needed to be done for this to work with VTK 8.2. At the time where OpenGLInitState() is called there is no active context, so a context need to be created first. Insert Superclass::OpenGLInitContext(); just be before the call to OpenGLInitiState. Perhaps, one could check if a context exist and is initialized and initialize if needed.

@JensMunkHansen I've actually been able to get rid of all that complexity by getting rid of vtkInternalOpenGLRenderWindow and using vtkExternalOpenGLRenderWindow from VTK.

@nocnokneo Similar to the how it is used in https://github.com/qCring/QuickVtk I guess. I managed to use QVTKInteractorAdapter and queue all QEvent's first and use vtkEventQtSlotConnect to connect StartEvent and EndEvent to Qt slots for easy integration. Someone should start a QMLGuiSupport add-on to VTK. Do you know of anything like that?

Someone should start a QMLGuiSupport add-on to VTK. Do you know of anything like that?

@JensMunkHansen I did see some activity on https://gitlab.kitware.com/vtk/vtk/-/issues/16061 recently. If you are able to chime in there with offers to support that effort, it could help it get some traction. I may be able to make some contributions, but can't commit yet.

@nocnokneo I did see that as well. They seemed frustrated about the changes to vtkExternalOpenGLRenderWindow and support for interactions. Would be nice if VTK did QML integration for someone and this was put into the VTK codebase. I have more or less ported the FourPaneViewer to QML using QVTKInteractorAdapter, but it only works when QSG is running "basic" and the reslicecallback is not working at the moment. I will join that thread

I use multi-threaded rendering with OpenGL in QML. I render videos using OpenGL in QML through it. I would like to know if it's possible to embed a PCL viewer in this OpenGL context.