Skip to content

Instantly share code, notes, and snippets.

@SteveRuben
Created October 21, 2015 08:30
Show Gist options
  • Star 66 You must be signed in to star a gist
  • Fork 26 You must be signed in to fork a gist
  • Save SteveRuben/2a15909e384b582c51b5 to your computer and use it in GitHub Desktop.
Save SteveRuben/2a15909e384b582c51b5 to your computer and use it in GitHub Desktop.
Multiple streaming in c++ using opencv; OpenCV video streaming over TCP/IP
/**
* OpenCV video streaming over TCP/IP
* Client: Receives video from server and display it
* by Steve Tuenkam
*/
#include "opencv2/opencv.hpp"
#include <sys/socket.h>
#include <arpa/inet.h>
#include <unistd.h>
using namespace cv;
int main(int argc, char** argv)
{
//--------------------------------------------------------
//networking stuff: socket , connect
//--------------------------------------------------------
int sokt;
char* serverIP;
int serverPort;
if (argc < 3) {
std::cerr << "Usage: cv_video_cli <serverIP> <serverPort> " << std::endl;
}
serverIP = argv[1];
serverPort = atoi(argv[2]);
struct sockaddr_in serverAddr;
socklen_t addrLen = sizeof(struct sockaddr_in);
if ((sokt = socket(PF_INET, SOCK_STREAM, 0)) < 0) {
std::cerr << "socket() failed" << std::endl;
}
serverAddr.sin_family = PF_INET;
serverAddr.sin_addr.s_addr = inet_addr(serverIP);
serverAddr.sin_port = htons(serverPort);
if (connect(sokt, (sockaddr*)&serverAddr, addrLen) < 0) {
std::cerr << "connect() failed!" << std::endl;
}
//----------------------------------------------------------
//OpenCV Code
//----------------------------------------------------------
Mat img;
img = Mat::zeros(480 , 640, CV_8UC1);
int imgSize = img.total() * img.elemSize();
uchar *iptr = img.data;
int bytes = 0;
int key;
//make img continuos
if ( ! img.isContinuous() ) {
img = img.clone();
}
std::cout << "Image Size:" << imgSize << std::endl;
namedWindow("CV Video Client",1);
while (key != 'q') {
if ((bytes = recv(sokt, iptr, imgSize , MSG_WAITALL)) == -1) {
std::cerr << "recv failed, received bytes = " << bytes << std::endl;
}
cv::imshow("CV Video Client", img);
if (key = cv::waitKey(10) >= 0) break;
}
close(sokt);
return 0;
}
/**
* OpenCV video streaming over TCP/IP
* Server: Captures video from a webcam and send it to a client
* by Isaac Maia
*/
#include "opencv2/opencv.hpp"
#include <iostream>
#include <sys/socket.h>
#include <arpa/inet.h>
#include <sys/ioctl.h>
#include <net/if.h>
#include <unistd.h>
#include <string.h>
using namespace cv;
void *display(void *);
int capDev = 0;
VideoCapture cap(capDev); // open the default camera
int main(int argc, char** argv)
{
//--------------------------------------------------------
//networking stuff: socket, bind, listen
//--------------------------------------------------------
int localSocket,
remoteSocket,
port = 4097;
struct sockaddr_in localAddr,
remoteAddr;
pthread_t thread_id;
int addrLen = sizeof(struct sockaddr_in);
if ( (argc > 1) && (strcmp(argv[1],"-h") == 0) ) {
std::cerr << "usage: ./cv_video_srv [port] [capture device]\n" <<
"port : socket port (4097 default)\n" <<
"capture device : (0 default)\n" << std::endl;
exit(1);
}
if (argc == 2) port = atoi(argv[1]);
localSocket = socket(AF_INET , SOCK_STREAM , 0);
if (localSocket == -1){
perror("socket() call failed!!");
}
localAddr.sin_family = AF_INET;
localAddr.sin_addr.s_addr = INADDR_ANY;
localAddr.sin_port = htons( port );
if( bind(localSocket,(struct sockaddr *)&localAddr , sizeof(localAddr)) < 0) {
perror("Can't bind() socket");
exit(1);
}
//Listening
listen(localSocket , 3);
std::cout << "Waiting for connections...\n"
<< "Server Port:" << port << std::endl;
//accept connection from an incoming client
while(1){
//if (remoteSocket < 0) {
// perror("accept failed!");
// exit(1);
//}
remoteSocket = accept(localSocket, (struct sockaddr *)&remoteAddr, (socklen_t*)&addrLen);
//std::cout << remoteSocket<< "32"<< std::endl;
if (remoteSocket < 0) {
perror("accept failed!");
exit(1);
}
std::cout << "Connection accepted" << std::endl;
pthread_create(&thread_id,NULL,display,&remoteSocket);
//pthread_join(thread_id,NULL);
}
//pthread_join(thread_id,NULL);
//close(remoteSocket);
return 0;
}
void *display(void *ptr){
int socket = *(int *)ptr;
//OpenCV Code
//----------------------------------------------------------
Mat img, imgGray;
img = Mat::zeros(480 , 640, CV_8UC1);
//make it continuous
if (!img.isContinuous()) {
img = img.clone();
}
int imgSize = img.total() * img.elemSize();
int bytes = 0;
int key;
//make img continuos
if ( ! img.isContinuous() ) {
img = img.clone();
imgGray = img.clone();
}
std::cout << "Image Size:" << imgSize << std::endl;
while(1) {
/* get a frame from camera */
cap >> img;
//do video processing here
cvtColor(img, imgGray, CV_BGR2GRAY);
//send processed image
if ((bytes = send(socket, imgGray.data, imgSize, 0)) < 0){
std::cerr << "bytes = " << bytes << std::endl;
break;
}
}
}
@mandrelbrotset
Copy link

You did not include opencv4 and pthread libraries while compiling.
You can use the makefile in my previous post. Copy it into a file Makefile. Then rename your filename to srv.cpp.
Run make server in terminal and you should get server.

it is'nt working
Screenshot from 2019-10-14 23-20-52

Can you run this pkg-config opencv4 --cflags --libs . Also do you know the version of opencv you're using.

yes i am using opencv 4.1.1 .
Screenshot from 2019-10-14 23-29-45

It appears that your opencv installation was not configured properly. Did you build opencv from source or you installed it using sudo apt-get?

@mahikaajain
Copy link

You did not include opencv4 and pthread libraries while compiling.
You can use the makefile in my previous post. Copy it into a file Makefile. Then rename your filename to srv.cpp.
Run make server in terminal and you should get server.

it is'nt working
Screenshot from 2019-10-14 23-20-52

Can you run this pkg-config opencv4 --cflags --libs . Also do you know the version of opencv you're using.

yes i am using opencv 4.1.1 .
Screenshot from 2019-10-14 23-29-45

It appears that your opencv installation was not configured properly. Did you build opencv from source or you installed it using sudo apt-get?

i just extracted the opencv.sh file using chmod +x opencv.sh
version="$(wget -q -O - http://sourceforge.net/projects/opencvlibrary/files/opencv-unix | egrep -m1 -o '"0-9+' | cut -c2-)"
echo "Installing OpenCV" $version
mkdir OpenCV
cd OpenCV
echo "Removing any pre-installed ffmpeg and x264"
echo "sudo apt-get remove x264 libx264-dev"
echo ""
echo "Installing Dependenices"
sudo apt-get install libopencv-dev
echo "Build Tools
"
echo "<----------------------------------------------------Build Tools------------------------------------------------->"
sudo apt-get install build-essential checkinstall cmake pkg-config
echo "_*_*_*_*_"
echo "<------------------------------------------------------Image I/O----------------------------------------------------->"
sudo apt-get install libtiff5-dev libjpeg-dev libjasper-dev libpng-dev zliblg-dev libwebp-dev libopenexr-dev libgdal-dev
echo "
"
echo "<--------------------------------------------------------Video I/O--------------------------------------------------->"
sudo apt-get install libavcodec-dev libavformat-dev libmp3lame-dev
sudo apt-get install libswscale-dev libdc1394-22-dev libxine-dev
sudo apt-get install libgstreamer0.10-dev libgstreamer-plugins-base0.10-dev
sudo apt-get install libv4l-dev v4l-utils libfaac-dev libopencore-amrnb-dev
sudo apt-get instal libopencore-amrwb-dev libtheora-dev libvorbis-dev
sudo apt-get insta libxvidcore-dev libx264-dev x264 yasm
echo "
"
echo "Parallelism and linear algebra libraries"
sudo apt-get install libtbb-dev libeigen3-dev
echo "
"
echo "<------------------------------------------------for GUI------------------------------------------------->"
sudo apt-get install libqt4-dev libgtk2.0-dev qt5-default
echo " sudo apt-get install libvtk6-dev"
echo "*************************************************************************************************************"
echo "<--------------For JAVA-------------------->"
echo sudo apt-get install ant default-jdk
echo "<-------------For Python------
-------->"
echo sudo apt-get install python-dev python-tk python-numpy python3-dev python3-tk python3-numpy python-matplotlib
sudo apt-get install python-opencv
echo "%%%%%%%%%%%%%%%%%%%%%%"
echo "Downloading OpenCV" $version
wget -O OpenCV-$version.zip http://sourceforge.net/projects/opencvlibrary/files/opencv-unix/$version/opencv-"$version".zip/download
echo "Installing OpenCV" $version
unzip OpenCV-$version.zip
cd opencv-$version
mkdir build
cd build
echo "*
_*_*_*_*_*_*"
cmake -D CMAKE_BUILD_TYPE=RELEASE
-D CMAKE_INSTALL_PREFIX=/usr/local
-D WITH_TBB=ON
-D BUILD_NEW_PYTHON_SUPPORT=ON
-D WITH_V4L=ON
-D BUILD_opencv_java=ON
-D INSTALL_C_EXAMPLES=ON
-D INSTALL_PYTHON_EXAMPLES=ON
-D BUILD_DOCS=ON
-D BUILD_EXAMPLES=ON
-D WITH_QT=ON
-D WITH_OPENGL=ON
-D WITH_EIGEN=ON ..
make -j4
echo "
"
sudo make install
echo "
"
sudo sh -c 'echo "/usr/local/lib" > /etc/ld.so.conf.d/opencv.conf'
echo "
******************************"
sudo ldconfig
echo "OpenCV" $version "ready to be used"

which included all these commands .

@mahikaajain
Copy link

You did not include opencv4 and pthread libraries while compiling.
You can use the makefile in my previous post. Copy it into a file Makefile. Then rename your filename to srv.cpp.
Run make server in terminal and you should get server.

it is'nt working
Screenshot from 2019-10-14 23-20-52

Can you run this pkg-config opencv4 --cflags --libs . Also do you know the version of opencv you're using.

yes i am using opencv 4.1.1 .
Screenshot from 2019-10-14 23-29-45

It appears that your opencv installation was not configured properly. Did you build opencv from source or you installed it using sudo apt-get?

i just extracted the opencv.sh file using chmod +x opencv.sh
version="$(wget -q -O - http://sourceforge.net/projects/opencvlibrary/files/opencv-unix | egrep -m1 -o '"0-9+' | cut -c2-)"
echo "Installing OpenCV" $version
mkdir OpenCV
cd OpenCV
echo "Removing any pre-installed ffmpeg and x264"
echo "sudo apt-get remove x264 libx264-dev"
echo "_" echo "Installing Dependenices" sudo apt-get install libopencv-dev echo "Build Tools" echo "<----------------------------------------------------Build Tools------------------------------------------------->" sudo apt-get install build-essential checkinstall cmake pkg-config echo "__________________" echo "<------------------------------------------------------Image I/O----------------------------------------------------->" sudo apt-get install libtiff5-dev libjpeg-dev libjasper-dev libpng-dev zliblg-dev libwebp-dev libopenexr-dev libgdal-dev echo "************************" echo "<--------------------------------------------------------Video I/O--------------------------------------------------->" sudo apt-get install libavcodec-dev libavformat-dev libmp3lame-dev sudo apt-get install libswscale-dev libdc1394-22-dev libxine-dev sudo apt-get install libgstreamer0.10-dev libgstreamer-plugins-base0.10-dev sudo apt-get install libv4l-dev v4l-utils libfaac-dev libopencore-amrnb-dev sudo apt-get instal libopencore-amrwb-dev libtheora-dev libvorbis-dev sudo apt-get insta libxvidcore-dev libx264-dev x264 yasm echo "" echo "Parallelism and linear algebra libraries" sudo apt-get install libtbb-dev libeigen3-dev echo "" echo "<------------------------------------------------for GUI------------------------------------------------->" sudo apt-get install libqt4-dev libgtk2.0-dev qt5-default echo " sudo apt-get install libvtk6-dev" echo "***********************************************************************************************" echo "<--------------For JAVA-------------------->" echo sudo apt-get install ant default-jdk echo "<-------******------For Python------__-------->" echo sudo apt-get install python-dev python-tk python-numpy python3-dev python3-tk python3-numpy python-matplotlib sudo apt-get install python-opencv echo "%%%%%%%%%%%%%%%%%%%%%%" echo "Downloading OpenCV" $version wget -O OpenCV-$version.zip http://sourceforge.net/projects/opencvlibrary/files/opencv-unix/$version/opencv-"$version".zip/download echo "Installing OpenCV" $version unzip OpenCV-$version.zip cd opencv-$version mkdir build cd build echo "_____*_____*_" cmake -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/local -D WITH_TBB=ON -D BUILD_NEW_PYTHON_SUPPORT=ON -D WITH_V4L=ON -D BUILD_opencv_java=ON -D INSTALL_C_EXAMPLES=ON -D INSTALL_PYTHON_EXAMPLES=ON -D BUILD_DOCS=ON -D BUILD_EXAMPLES=ON -D WITH_QT=ON -D WITH_OPENGL=ON -D WITH_EIGEN=ON .. make -j4 echo "_" sudo make install echo "_" sudo sh -c 'echo "/usr/local/lib" > /etc/ld.so.conf.d/opencv.conf' echo "_***************************"
sudo ldconfig
echo "OpenCV" $version "ready to be used"

which included all these commands .

Screenshot from 2019-10-14 23-48-55

@mandrelbrotset
Copy link

Awesome, change line 3 in the Makefile to OPENCV = 'pkg-config opencv --cflags --libs'.
Hopefully it works this time!

@mahikaajain
Copy link

Awesome, change line 3 in the Makefile to OPENCV = 'pkg-config opencv --cflags --libs'.
Hopefully it works this time!

nope it is'nt working :(
Screenshot from 2019-10-15 00-02-41

@mandrelbrotset
Copy link

Can I see your Makefile?

@mahikaajain
Copy link

Can I see your Makefile?

I resolved that, the error was because of the quotes i was putting in that line .Now this output is coming:
Screenshot from 2019-10-15 00-11-16

Screenshot from 2019-10-15 00-11-54

@mandrelbrotset
Copy link

Another alternative is to do g++ srv.cpp -lpthread + paste result of pkg-config here
Then run.

@mahikaajain
Copy link

which pkg-config?

Another alternative is to do g++ srv.cpp -lpthread + paste result of pkg-config here
Then run.

@mandrelbrotset
Copy link

which pkg-config?

Another alternative is to do g++ srv.cpp -lpthread + paste result of pkg-config here
Then run.

Run pkg-config opencv --cflags --libs
Run g++ srv.cpp -lpthread + result of above command

@mahikaajain
Copy link

it is working thankyou so much for your help.

Another alternative is to do g++ srv.cpp -lpthread + paste result of pkg-config here
Then run.

@mandrelbrotset
Copy link

You're welcome!

@mahikaajain
Copy link

this code will work on different machine or also on local server?

@SteveRuben
Copy link
Author

SteveRuben commented Oct 15, 2019 via email

@SteveRuben
Copy link
Author

thank's to @sheriffolaoye and @mahika2

@mahikaajain
Copy link

It is not working on different machines and the camera is'nt opening on client side for capturing the video .what to do

@mandrelbrotset
Copy link

It is not working on different machines and the camera is'nt opening on client side for capturing the video .what to do

Hi @mahika2,

Looking at the code, it’s only the server that accesses the web camera. The client only displays frames sent from the server.

@mahikaajain
Copy link

Yes sorry on the server side the camera is'nt opening.when i run the code web camera light gets on for 1 sec without opening of camera and then turns off. And there is nothing recorded. Atlast its also not working on different machines idk why.

@mandrelbrotset
Copy link

Yes sorry on the server side the camera is'nt opening.when i run the code web camera light gets on for 1 sec without opening of camera and then turns off. And there is nothing recorded. Atlast its also not working on different machines idk why.

Did you modify the server code? From your previous screenshots it seems you’re running Ubuntu in virtualbox, are you sure ubuntu has full access to your webcam?

@mahikaajain
Copy link

mahikaajain commented Oct 18, 2019

no i didn't modify the code.for other codes the camera is working for example for capturing video through opencv the camera works fine.

@ujjwalgoyal65
Copy link

image
please tell me the solution for this error

@ujjwalgoyal65
Copy link

it runs only for single thread and then exit.

@mandrelbrotset
Copy link

Yes sorry on the server side the camera is'nt opening.when i run the code web camera light gets on for 1 sec without opening of camera and then turns off. And there is nothing recorded. Atlast its also not working on different machines idk why.

For it to work on different machines, instead of using 127.0.0.1(localhost), you have to use the ip address of the machine running the server.

@mandrelbrotset
Copy link

image
please tell me the solution for this error

Hi @ujjwalgoyal65,

Those are just warnings, you can ignore them. C++ just doesn't like the way the variable is being converted.

@mandrelbrotset
Copy link

Screenshot from 2019-10-18 22-27-21
I installed opencv4.1.1 and everything works for me.
At this point I'm not sure what's wrong.

@ujjwalgoyal65
Copy link

can u plz mail me the client and server code .
emial id:- ujjwalgoyal65@gmail.com

@BOUROUINA
Copy link

BOUROUINA commented Jun 2, 2020

Hi everyone
please I have a question regarding video savage at the client level, that it is sent from a server. You can tell me which lines of code to add

@SteveRuben
Copy link
Author

Hi everyone
please I have a question regarding video savage at the client level, that it is sent from a server. You can tell me which lines of code to add

*void send_data(void ptr)

@nonlinearjunkie
Copy link

this code work very well , but when one thread end , the streaming stop for all others threads

Can you share the makefile or CMAkeLists.txt file to run the above scripts?

@Anjaneyakarjigi
Copy link

Hi,
thanks for your code, to change this code to UDP, what should we do?

just replace SOCK_STREAM to SOCK_DGRAM, It works I tested
Thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment