VideoCapture returns empty Mat and imshow() crashes. [closed]
I have following code (in ROS), where I read an Image stream from a webcam, convert it and send it to another ROS program, and display it:
#include <ros/ros.h>
#include <image_transport/image_transport.h>
#include <opencv2/highgui/highgui.hpp>
#include <opencv2/imgproc/imgproc.hpp>
#include <cv_bridge/cv_bridge.h>
#include <thread>
#include <chrono>
using namespace std;
int main(int argc, char** argv) {
ros::init(argc, argv, "image_publisher");
ros::NodeHandle nh;
image_transport::ImageTransport it(nh);
image_transport::Publisher pub = it.advertise("camera/image", 1);
cv::VideoCapture cap;
cap.open(0);
// give webcam some time
std::this_thread::sleep_for(std::chrono::milliseconds(200));
// Check if video device can be opened with the given index
if (!cap.isOpened())
return 1;
sensor_msgs::ImagePtr msg;
ROS_INFO("Webcam publisher running");
ros::Rate loop_rate(10);
while (nh.ok()) {
cv::Mat frame;
cap >> frame;
cout<<"frame type: "<<frame.type()<<"\n";
cout<<"frame empty: "<<frame.empty()<<"\n";
// Check if grabbed frame is actually full with some content
if (!frame.empty()) {
msg = cv_bridge::CvImage(std_msgs::Header(), "rgb8", frame).toImageMsg();
pub.publish(msg);
cv::imshow("Webcam",frame);
cv::waitKey(0);
}
ros::spinOnce();
loop_rate.sleep();
}
}
My problem is that the code always outputs "frame empty: 0". The if condition if(!frame.empty())
is nevertheless entered and the ROS msg (created from the frame) which is sent, is just fine. The subscriber to this ROS msg has no problem to read the frame and show it with imshow()
. (For those who aren't familiar with ROS: I send the frame to another program where I can read and display it without any problems).
I need to display the frame already in the code above, but it always crashes at cv::imshow("Webcam",frame);
, giving the following debugging (with gdb) output:
Program received signal SIGSEGV, Segmentation fault.
0x00007ffff760cd20 in cv::_InputArray::size(int) const ()
from /usr/lib/x86_64-linux-gnu/libopencv_core.so.2.4
(gdb) bt
#0 0x00007ffff760cd20 in cv::_InputArray::size(int) const ()
from /usr/lib/x86_64-linux-gnu/libopencv_core.so.2.4
#1 0x00007ffff66e4887 in cv::imshow(cv::String const&, cv::_InputArray const&) () from /usr/local/lib/libopencv_highgui.so.3.0
#2 0x00000000004023c8 in main (argc=1, argv=<optimized out>)
at /ibox/ros/catkin_ws/src/ros/webcam_publisher/src/webcam_publisher.cpp:41
(gdb)
If I comment the lines cv::imshow("Webcam",frame);
(and cv::waitKey(0);
), everything works just fine. I am using opencv 3.
Does anybody have a clue what could be going on here?
please check the return value from
cap.open(0)
. if it returns false, your camera was not found/not useable.also check
frame.empty()
, when reading. throwing an invalid/empty Mat at imshow() will crash.cap.open(0)
returns 1. As I stated above,frame.empty()
returns 0. BUT the strange thing is, thatframe.type()
returns 16 and if I comment imshow(), the code works fine andmsg = cv_bridge::CvImage(std_msgs::Header(), "rgb8", frame).toImageMsg();
actually sends a valid frame, which I can read and display without any problems.i bet , it all works fine, if you throw out all ROS code.
can it be, that your ROS msg is somehow transfering pixel ownership, thus invalidating the Mat frame ?
I thought about this as well, but removing all ROS code still does not change anything. I will leave this annoying problem for now and will update this question once I have figured it out. But thanks anyway!
Have you tried to display the image first and the do the rest?
yes, didn't work neither..
So you are saying that frame is not empty and the imshow is throwing an error?
But your error seems to be here:
/ibox/ros/catkin_ws/src/ros/webcam_publisher/src/webcam_publisher.cpp:41
Have you tried it with a still image and imread; like here?Isn't it because you have 2 OpenCV installed:
? core 2.4 and highgui 3.0
I installed OpenCV following this tutorial (with OpenCV 3.0.0-rc1 instead of OpenCV 3.0.0 alpha). I never installed anything else regarding OpenCV..
ahh, you installed opencv3, ros installed 2.4 under the hood ?
it all would make pretty much sense then, ros might be pulling in different so's than your main prog, resulting in the most weird bugs imaginable..
aha! I didn't know that ROS actually installs OpenCV as well! hmm.. this certainly is annoying. But thanks for the hint, very helpful!