Ask Your Question
0

Assertion fail when merging channels in OpenCV

asked 2016-03-11 12:27:13 -0500

jsanarde gravatar image

I am working on a code that receives a video feed and edits it so that the video displayed has been altered in a way that the user sees what a colorblind person sees, the code is done but i am having issues merging the channels into a final frame. Here is the code i am using:

 /*
#include <opencv2/highgui/highgui.hpp>
#include "opencv2/imgproc/imgproc.hpp"
#include "opencv2/core/utility.hpp"
#include <opencv2\core\core.hpp>
#include "iostream"
*/

using namespace cv; using namespace std;

int main(int argc, const char** argv) { VideoCapture cap(0); if (!cap.isOpened()) return -1; cap.set(CV_CAP_PROP_FRAME_HEIGHT, 400); cap.set(CV_CAP_PROP_FRAME_WIDTH, 400);

for (;;){

    Mat frame;
    cap >> frame;

    Mat src = frame.clone();

    vector<Mat> spl;
    split(src, spl);

    // Create an zero pixel image for filling purposes - will become clear later
    // Also create container images for B G R channels as colour images
    Mat empty_image = Mat::zeros(src.rows, src.cols, CV_8UC1);
    Mat for_the_sake_of_safety = Mat::zeros(src.rows, src.cols, CV_32FC3);
    Mat result_blue(src.rows, src.cols, CV_8UC3); // notice the 3 channels here!
    Mat result_green(src.rows, src.cols, CV_8UC3); // notice the 3 channels here!
    Mat result_red(src.rows, src.cols, CV_8UC3); // notice the 3 channels here!

    Mat fresh = Mat::zeros(src.rows, src.cols, CV_8UC3);

    // Create blue channel
    Mat in1[] = { spl[0], empty_image, empty_image };
    int from_to1[] = { 0, 0, 1, 1, 2, 2 };
    mixChannels(in1, 3, &result_blue, 1, from_to1, 3);

    // Create green channel 
    Mat in2[] = { empty_image, spl[1], empty_image };
    int from_to2[] = { 0, 0, 1, 1, 2, 2 };
    mixChannels(in2, 3, &result_green, 1, from_to2, 3);

    // Create red channel
    Mat in3[] = { empty_image, empty_image, spl[2] };
    int from_to3[] = { 0, 0, 1, 1, 2, 2 };
    mixChannels(in3, 3, &result_red, 1, from_to3, 3);

    result_blue.convertTo(result_blue, CV_32FC3);
    result_green.convertTo(result_green, CV_32FC3);
    result_red.convertTo(result_red, CV_32FC3);

    float tempBGR[3];
    float L, M, S, LShift, MShift, SShift;

    for (int i = 0; i < src.rows; i++){
        for (int j = 0; j < src.cols * 3; j++){
            // here is wher i define a new sub matrix
            tempBGR[0] = result_blue.at<float>(i, j) / 255; // for scale
            tempBGR[1] = result_green.at<float>(i, j) / 255;
            tempBGR[2] = result_red.at<float>(i, j) / 255;

            //could we alternatively make a separate float matrix to store things and then convert to Mat format
            L = 17.8824 * tempBGR[2] + 43.5161 * tempBGR[1] + 4.1193 * tempBGR[0];
            M = 3.45570 * tempBGR[2] + 27.1554 * tempBGR[1] + 3.8671 * tempBGR[0];
            S = 0.02996 * tempBGR[2] + 0.18431 * tempBGR[1] + 1.4670 * tempBGR[0];

            // LMS to L'M'S' {Tritanopia}
            LShift = 1 * L + 0 * M + 0 * S;
            MShift = 0 * L + 1 * M + 0 * S;
            SShift = -0.395913 * L + 0.801109 * M + 0 * S;

            // l'M'S' to BGR
            tempBGR[2] = 0.080944942 * LShift - 0.130505254 * MShift + 0.116728267 * SShift;
            tempBGR[1] = -0.010248719 * LShift + 0.05401967 * MShift - 0.11362094 * SShift;
            tempBGR[0] = -0.000365487 * LShift - 0.004121628 * MShift + 0.693554394 * SShift;

            Mat tempMatSChanB = Mat(1, 3, CV_32FC1, tempBGR[0]);
            Mat tempMatSChanG = Mat(1, 3, CV_32FC1, tempBGR[1]);
            Mat tempMatSChanR = Mat(1, 3 ...
(more)
edit retag flag offensive close merge delete

Comments

1

Well, the Mats you are trying to copy from are one channel, and you are trying to copy into a 3 channel Mat.

Change the type to CV_32FC1 and use the copys, or change the size of tempMatMChannI to (1,3) and use cv::merge(tempBGR, 3, tempMatMChannI). Which one depends on the the precise details. I think you want the second, using merge.

I should also point out that the store in src bit only copies one channel. Replace the uchar with

Vec<uchar, 3>

to get all three channels.

Tetragramm gravatar imageTetragramm ( 2016-03-11 22:25:01 -0500 )edit

After adjusting the channels the error still lies on merge, the problem is: the argument list does not match (float[3], int, cv::Mat )

jsanarde gravatar imagejsanarde ( 2016-03-12 04:11:11 -0500 )edit

1 answer

Sort by ยป oldest newest most voted
1

answered 2016-03-12 16:35:11 -0500

Tetragramm gravatar image

updated 2016-03-14 07:09:42 -0500

Oh, I see, I didn't check the type and mis-understood what you were trying to do.

Ok, you are doing the same set of operations to each pixel, so you should instead do that to the entire Mat.

You can replace all of this with a few lines of code.

 for (;;) 
{
    Mat frame;
    cap >> frame;

    Mat src = frame.clone();

    vector<Mat> spl;
    split(src, spl);

    Mat tempBGR[3];
    Mat L, M, S;
    Mat LMSShift[3];

    //Converts to float matrices and divides by 255
    spl[0].convertTo(tempBGR[0], CV_32F, 1.0 / 255.0);
    spl[1].convertTo(tempBGR[1], CV_32F, 1.0 / 255.0);
    spl[2].convertTo(tempBGR[2], CV_32F, 1.0 / 255.0);


    //L = 17.8824 * tempBGR[2] + 43.5161 * tempBGR[1] + 4.1193 * tempBGR[0];
    //M = 3.45570 * tempBGR[2] + 27.1554 * tempBGR[1] + 3.8671 * tempBGR[0];
    //S = 0.02996 * tempBGR[2] + 0.18431 * tempBGR[1] + 1.4670 * tempBGR[0];
    //L
    addWeighted(tempBGR[0], 4.1193, tempBGR[1], 43.5161, 0, L);
    addWeighted(L, 1, tempBGR[2], 17.8824, 0, L);
    //M
    addWeighted(tempBGR[0], 3.8671, tempBGR[1], 27.1554, 0, M);
    addWeighted(M, 1, tempBGR[2], 3.45570, 0, M);
    //S (not used here)
    addWeighted(tempBGR[0], 1.4670, tempBGR[1], 0.18431, 0, S);
    addWeighted(S, 1, tempBGR[2], 0.02996, 0, S);

    // LMS to L'M'S' {Tritanopia}
    //LShift = 1 * L + 0 * M + 0 * S;
    //MShift = 0 * L + 1 * M + 0 * S;
    //SShift = -0.395913 * L + 0.801109 * M + 0 * S;
    LMSShift[0] = L;
    LMSShift[1] = M;
    addWeighted(L, -0.395913, M, 0.801109, 0, LMSShift[2]);

    // l'M'S' to BGR
    //tempBGR[2] = 0.080944942 * LShift - 0.130505254 * MShift + 0.116728267 * SShift;
    //tempBGR[1] = -0.010248719 * LShift + 0.05401967 * MShift - 0.11362094 * SShift;
    //tempBGR[0] = -0.000365487 * LShift - 0.004121628 * MShift + 0.693554394 * SShift;
    //tempBGR[0]
    addWeighted(LMSShift[0], 0.080944942, LMSShift[1], -0.130505254, 0, tempBGR[0]);
    addWeighted(tempBGR[0], 1, LMSShift[2], 0.116728267, 0, tempBGR[0]);
    //tempBGR[1]
    addWeighted(LMSShift[0], -0.010248719, LMSShift[1], 0.05401967, 0, tempBGR[1]);
    addWeighted(tempBGR[1], 1, LMSShift[2], -0.11362094, 0, tempBGR[1]);
    //tempBGR[2]
    addWeighted(LMSShift[0], -0.000365487, LMSShift[1], -0.004121628, 0, tempBGR[2]);
    addWeighted(tempBGR[2], 1, LMSShift[2], 0.693554394, 0, tempBGR[2]);

    merge(tempBGR, 3, src);
    src.convertTo(src, CV_8UC3, 255);

    imshow("conversion", src);

    if (waitKey(30) >= 0) break;
}

Does this make sense to you? Why it's done this way instead of looping over every pixel?

edit flag offensive delete link more

Comments

Thank you very much for your answer :-), I am not sure on how the addWeighted function works, but trying out the code gives me a purple filter I have tried a different approach using the principles from my previous code and it got me closer to an approximation of the final solution, here is the code:

tempBGR[0] = frame.at<Vec3b>(i, j)[0];
tempBGR[1] = frame.at<Vec3b>(i, j)[1];
tempBGR[2] = frame.at<Vec3b>(i, j)[2];

i did calculations on them and then i reassembled them back using the same line backwards

jsanarde gravatar imagejsanarde ( 2016-03-13 15:57:12 -0500 )edit

Oh, well probably because I missed a negative sign when computing what was SShift. That last add weighted. I'm glad you got it working though.

Tetragramm gravatar imageTetragramm ( 2016-03-13 16:34:41 -0500 )edit

thank you once again for your solution, upon testing it with the modified values the output is a video feed with a blue filter

jsanarde gravatar imagejsanarde ( 2016-03-13 19:29:48 -0500 )edit

Feel free to call me an idiot. I forgot the whole LMS to BGR section at the end. Whoops. Edited my answer.

Tetragramm gravatar imageTetragramm ( 2016-03-13 22:21:05 -0500 )edit

A million thanks again, it seems your way is less resource intensive on my camera and it doesn't have noise in it, there is a small mistake

merge(tempBGR, 3, src);
image.convertTo(src, CV_8UC3, 255);

at the end you are merging back into src, image is not defined

jsanarde gravatar imagejsanarde ( 2016-03-14 04:25:12 -0500 )edit
1

The reason you would want to do it this way is because OpenCV has optimizations for SSE and AVX instructions, which allow your processor to do the same thing to multiple pixels at the same time. So while it does take more memory, it's still significantly faster.

Before you do your next project, take a quick look through the basic math operations. The basic things, and many of the more advanced image processing techniques are already written, to save you the trouble.

Tetragramm gravatar imageTetragramm ( 2016-03-14 07:08:22 -0500 )edit

That makes more sense

jsanarde gravatar imagejsanarde ( 2016-03-14 08:46:06 -0500 )edit

Question Tools

1 follower

Stats

Asked: 2016-03-11 12:27:13 -0500

Seen: 1,745 times

Last updated: Mar 14 '16