VideoCapture returns empty Mat and imshow() crashes.

asked 2015-10-07 03:27:40 -0500

takahashi gravatar image

I have following code (in ROS), where I read an Image stream from a webcam, convert it and send it to another ROS program, and display it:

#include <ros/ros.h>
#include <image_transport/image_transport.h>
#include <opencv2/highgui/highgui.hpp>
#include <opencv2/imgproc/imgproc.hpp>
#include <cv_bridge/cv_bridge.h>
#include <thread>
#include <chrono>

using namespace std;

int main(int argc, char** argv) {

    ros::init(argc, argv, "image_publisher");
    ros::NodeHandle nh;
    image_transport::ImageTransport it(nh);
    image_transport::Publisher pub = it.advertise("camera/image", 1);

    cv::VideoCapture cap;
    cap.open(0);
    // give webcam some time
    std::this_thread::sleep_for(std::chrono::milliseconds(200));
    // Check if video device can be opened with the given index
    if (!cap.isOpened())
        return 1;
    sensor_msgs::ImagePtr msg;
    ROS_INFO("Webcam publisher running");

    ros::Rate loop_rate(10); 
    while (nh.ok()) {
        cv::Mat frame;
        cap >> frame;
        cout<<"frame type: "<<frame.type()<<"\n";
            cout<<"frame empty: "<<frame.empty()<<"\n";
        // Check if grabbed frame is actually full with some content
        if (!frame.empty()) {
            msg = cv_bridge::CvImage(std_msgs::Header(), "rgb8", frame).toImageMsg();
            pub.publish(msg);
            cv::imshow("Webcam",frame);
            cv::waitKey(0);
        }
        ros::spinOnce();
        loop_rate.sleep();
    }
}

My problem is that the code always outputs "frame empty: 0". The if condition if(!frame.empty()) is nevertheless entered and the ROS msg (created from the frame) which is sent, is just fine. The subscriber to this ROS msg has no problem to read the frame and show it with imshow(). (For those who aren't familiar with ROS: I send the frame to another program where I can read and display it without any problems).

I need to display the frame already in the code above, but it always crashes at cv::imshow("Webcam",frame);, giving the following debugging (with gdb) output:

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff760cd20 in cv::_InputArray::size(int) const ()
   from /usr/lib/x86_64-linux-gnu/libopencv_core.so.2.4
(gdb) bt
#0  0x00007ffff760cd20 in cv::_InputArray::size(int) const ()
   from /usr/lib/x86_64-linux-gnu/libopencv_core.so.2.4
#1  0x00007ffff66e4887 in cv::imshow(cv::String const&, cv::_InputArray const&) () from /usr/local/lib/libopencv_highgui.so.3.0
#2  0x00000000004023c8 in main (argc=1, argv=<optimized out>)
    at /ibox/ros/catkin_ws/src/ros/webcam_publisher/src/webcam_publisher.cpp:41
(gdb)

If I comment the lines cv::imshow("Webcam",frame); (and cv::waitKey(0);), everything works just fine. I am using opencv 3.

Does anybody have a clue what could be going on here?

edit retag flag offensive close merge delete

Comments

1

please check the return value from cap.open(0) . if it returns false, your camera was not found/not useable.

also check frame.empty() , when reading. throwing an invalid/empty Mat at imshow() will crash.

berak gravatar imageberak ( 2015-10-07 03:30:24 -0500 )edit
3

cap.open(0) returns 1. As I stated above, frame.empty() returns 0. BUT the strange thing is, that frame.type() returns 16 and if I comment imshow(), the code works fine and msg = cv_bridge::CvImage(std_msgs::Header(), "rgb8", frame).toImageMsg(); actually sends a valid frame, which I can read and display without any problems.

takahashi gravatar imagetakahashi ( 2015-10-07 03:48:38 -0500 )edit
1

i bet , it all works fine, if you throw out all ROS code.

can it be, that your ROS msg is somehow transfering pixel ownership, thus invalidating the Mat frame ?

berak gravatar imageberak ( 2015-10-07 04:12:31 -0500 )edit

I thought about this as well, but removing all ROS code still does not change anything. I will leave this annoying problem for now and will update this question once I have figured it out. But thanks anyway!

takahashi gravatar imagetakahashi ( 2015-10-07 04:41:23 -0500 )edit
1

Have you tried to display the image first and the do the rest?

        cv::imshow("Webcam",frame);
        msg = cv_bridge::CvImage(std_msgs::Header(), "rgb8", frame).toImageMsg();
        pub.publish(msg);
        cv::waitKey(0);
thdrksdfthmn gravatar imagethdrksdfthmn ( 2015-10-07 09:29:09 -0500 )edit

yes, didn't work neither..

takahashi gravatar imagetakahashi ( 2015-10-08 09:52:08 -0500 )edit

So you are saying that frame is not empty and the imshow is throwing an error?

thdrksdfthmn gravatar imagethdrksdfthmn ( 2015-10-08 10:03:29 -0500 )edit

But your error seems to be here: /ibox/ros/catkin_ws/src/ros/webcam_publisher/src/webcam_publisher.cpp:41 Have you tried it with a still image and imread; like here?

thdrksdfthmn gravatar imagethdrksdfthmn ( 2015-10-08 10:07:14 -0500 )edit

Isn't it because you have 2 OpenCV installed:

from /usr/lib/x86_64-linux-gnu/libopencv_core.so.2.4
#1  0x00007ffff66e4887 in cv::imshow(cv::String const&, cv::_InputArray const&) () from /usr/local/lib/libopencv_highgui.so.3.0

? core 2.4 and highgui 3.0

thdrksdfthmn gravatar imagethdrksdfthmn ( 2015-10-08 10:12:32 -0500 )edit

I installed OpenCV following this tutorial (with OpenCV 3.0.0-rc1 instead of OpenCV 3.0.0 alpha). I never installed anything else regarding OpenCV..

takahashi gravatar imagetakahashi ( 2015-10-09 01:53:34 -0500 )edit