An Easy way to output opencv mat as rtsp on ubuntu using gstreamer rtsp server

opencv_to_rtsp.md

An Easy way to output opencv mat as rtsp on ubuntu using gstreamer rtsp server

Installation

1. Download the latest version of gstreamer and gstreamer plugins from https://gstreamer.freedesktop.org/src/ (1.14.2 for example)

wget https://gstreamer.freedesktop.org/src/gstreamer/gstreamer-1.14.2.tar.xz
wget https://gstreamer.freedesktop.org/src/gst-plugins-ugly/gst-plugins-ugly-1.14.2.tar.xz
wget https://gstreamer.freedesktop.org/src/gst-plugins-good/gst-plugins-good-1.14.2.tar.xz
wget https://gstreamer.freedesktop.org/src/gst-plugins-base/gst-plugins-base-1.14.2.tar.xz
wget https://gstreamer.freedesktop.org/src/gst-plugins-bad/gst-plugins-bad-1.14.2.tar.xz
wget https://gstreamer.freedesktop.org/src/gst-rtsp-server/gst-rtsp-server-1.14.2.tar.xz

2. Extract downloaded files using tar Jxvf filename.tar.xz
3. Install gstreamer and gstreamer plugins like this (gstreamer for example, other plugins can be install in the same way)

cd gstreamer-1.14.2/
sudo ./configure && make && make install

Notice: Some of the plugins may needs another plugin first. So if you got an error that says something hasn't install yet, then switch the install order.

4. Verify your installation by running test file gst-rtsp-server-1.14.2/examples/test-readme. You should be able to see something when connecting to rtsp://127.0.0.1:8554/test in VLC player.

cd gst-rtsp-server-1.14.2/examples
./test-readme

Debug (These are some problems I encountered when installing)

• If you have anaconda installed, you may be calling gst-launch-1.0 in anaconda. Make sure you are running the right gstreamer in /usr/local/bin/gst-launch-1.0 by running which gst-launch-1.0. If you are running gstreamer in anaconda, you can just simply remove the gst-launch-1.0 in anaconda. The same thing to gst-inspect-1.0.
• If you got some plugins getting blacklist, make sure you don't have other versions (like 1.8.3 or 0.10) of gstreamer installed by running dpkg -l|grep gst. You can check the blacklist by running gst-inspect-1.0 -b.
• If you are using ssh, make sure you have forward port 8554 to localhost. (ssh usrname@ip -L 8554:localhost:8554)

Codes

1. Sends cv::mat to udpsink in opencv cv::VideoWriter.
2. Takes the udpsrc and stream as rtsp server in gst_rtsp_server.

Opencv part (writing timestamp on mat for example)

Compile using g++ cv_test.cpp -o cv_test `pkg-config --cflags --libs opencv`

#include <iostream>
#include <opencv2/opencv.hpp>
#include <sys/time.h>
#include <sstream>
#include <unistd.h>

using namespace std;

int main(int argc, char** argv) {

    // second part of sender pipeline
    cv::VideoWriter writer;
    writer.open("appsrc ! videoconvert ! video/x-raw,format=YUY2,width=512,height=512,framerate=30/1 ! jpegenc ! rtpjpegpay name=pay0 pt=96 ! udpsink host=localhost port=6666"
                , 0, (double)30, cv::Size(512, 512), true);
    if (!writer.isOpened()) {
        printf("=ERR= can't create video writer\n");
        return -1;
    }

    while(true) {
        struct timeval tp;
        gettimeofday(&tp, NULL);
        long int ms = tp.tv_sec * 1000 + tp.tv_usec / 1000;

        // timestamp to string
        std::string number;
        std::stringstream strstream;
        strstream << ms;
        strstream >> number;

        cv::Mat img(512, 512, CV_8UC3, cv::Scalar(0));
        cv::putText(img, //target image
                    number, //timestamp text
                    cv::Point(10, img.rows / 2), //top-left position
                    cv::FONT_HERSHEY_DUPLEX,
                    1.0,
                    CV_RGB(118, 185, 0), //font color
                    2);
        writer << img;
        imwrite("test_img.jpg", img);
        cout << '\r' << number << flush;
    }
}

RTSP server part (modified from gst-rtsp-server-1.14.2/examples/test-readme)

Compile using gcc rtsp_server_test.c -o rtsp_server_test `pkg-config --cflags --libs gstreamer-rtsp-server-1.0 gstreamer-1.0`

#include <gst/gst.h>

#include <gst/rtsp-server/rtsp-server.h>

int
main (int argc, char *argv[])
{
  GMainLoop *loop;
  GstRTSPServer *server;
  GstRTSPMountPoints *mounts;
  GstRTSPMediaFactory *factory;

  gst_init (&argc, &argv);

  loop = g_main_loop_new (NULL, FALSE);

  /* create a server instance */
  server = gst_rtsp_server_new ();

  /* get the mount points for this server, every server has a default object
   * that be used to map uri mount points to media factories */
  mounts = gst_rtsp_server_get_mount_points (server);

  /* make a media factory for a test stream. The default media factory can use
   * gst-launch syntax to create pipelines. 
   * any launch line works as long as it contains elements named pay%d. Each
   * element with pay%d names will be a stream */
  factory = gst_rtsp_media_factory_new ();
  gst_rtsp_media_factory_set_launch (factory,
      "( udpsrc address=localhost port=6666 ! application/x-rtp, media=video, clock-rate=90000, encoding-name=JPEG, payload=26 ! rtpjpegdepay ! rtpjpegpay name=pay0 pt=96 )");

  gst_rtsp_media_factory_set_shared (factory, TRUE);

  /* attach the test factory to the /test url */
  gst_rtsp_mount_points_add_factory (mounts, "/test", factory);

  /* don't need the ref to the mapper anymore */
  g_object_unref (mounts);

  /* attach the server to the default maincontext */
  gst_rtsp_server_attach (server, NULL);

  /* start serving */
  g_print ("stream ready at rtsp://127.0.0.1:8554/test\n");
  g_main_loop_run (loop);

  return 0;
}

To make ip public or change port add the following codes before server attach

gst_rtsp_server_set_address(server, "0.0.0.0");	// ip
gst_rtsp_server_set_service(server, "8555");	// port

Run

After compiling, run both file at the same time. Then you should be able to see the output at rtsp://127.0.0.1:8554/test using VLC player.

Reference

• An useful introduction for gstreamer
• Gstreamer documentation

---

Any suggestions are welcome!