Ask Your Question

Use pre-trained cnn to classify 6-channel-mats

asked 2019-05-24 02:48:02 -0500

LuisK gravatar image

Hey there,

I have a pre trained cnn, which was trained on 6 channel inputs (created out of two bgr images of same size).

vector<Mat> inputs;
vector<Mat> temp;
Mat blob;
cv::split(image_1, inputs);
cv::split(image_2, temp);
inputs.insert(inputs.end(), temp.begin(), temp.end());
dnn::blobFromImages(inputs, blob, 1.0/255.0, m_patchSize, 0, true, false);
Mat out = net.forward();

The program crashes in line blobFromImages. What am I doing wrong? Is it possible to classify Mats with more than 3 channels?

edit retag flag offensive close merge delete



what does "6-channel Mat's" really mean here ? do you have a link to a repo / paper ? a prototxt ?

blobFromImages is meant to produce a "batch" of several images, not "channels" ( is that what you wanted ???)

berak gravatar imageberak ( 2019-05-24 02:52:24 -0500 )edit

I don't have a link or something, because I trained the model myself with keras/tensorflow for python. What I want to do: classify two images / consecutive frames (Don't ask me why). I trained the model by mixing these two images into one 6 channel matrix.

LuisK gravatar imageLuisK ( 2019-05-24 03:35:04 -0500 )edit

I don't have a link or something

but you'll have code to train it, right ? please also have a look here , you can't use the keras output directly, but you'll have to process it in a similar way (freezing the graph, applying transforms, weeding out unneeded consts, etc.

here's unfortunately, where the real work starts.

berak gravatar imageberak ( 2019-05-24 03:39:57 -0500 )edit

1 answer

Sort by ยป oldest newest most voted

answered 2019-05-24 03:28:46 -0500

berak gravatar image

updated 2019-05-24 05:33:04 -0500

untested (bc. we don't know your context), but you probably can't use dnn::blobFromImages here, but have to do it manually,

we're going to use the same trick from here:

Mat img1; // bgr
Mat img2; // also, same size.

int W = img1.cols;
int H = img1.rows;
int sz[] = {1, 6, H, W}; // 1 (batch)img, 6 channels
Mat blob(4, sz, CV_32F); // allocate data

vector<Mat> ch1, ch2; // original image channels

// setup channel pointers:

// 1st image goes into planes [0,1,2]
ch1.push_back(Mat(H, W, CV_32F, blop.ptr<float>(0,0));
ch1.push_back(Mat(H, W, CV_32F, blop.ptr<float>(0,1));
ch1.push_back(Mat(H, W, CV_32F, blop.ptr<float>(0,2));
cv::split(img1, ch1); // this will handle the actual copy into our blob !)

// 2nd image goes into planes [3,4,5]
ch2.push_back(Mat(H, W, CV_32F, blop.ptr<float>(0,3));
ch2.push_back(Mat(H, W, CV_32F, blop.ptr<float>(0,4));
ch2.push_back(Mat(H, W, CV_32F, blop.ptr<float>(0,5));
cv::split(img2, ch2);

you'll have to apply your own scaling / resizing here (and order matters !)

edit flag offensive delete link more
Login/Signup to Answer

Question Tools

1 follower


Asked: 2019-05-24 02:48:02 -0500

Seen: 35 times

Last updated: May 24