Ask Your Question
0

is it possible to apply convolution in multiple dimensions?

asked 2016-01-08 05:20:57 -0500

theodore gravatar image

I would like to know if there is any function to apply convolution in more than 2 dimensions in Opencv. I know that there is the filter2d() but this one is limited in 2d right? So I was wondering if there is something similar to convn in matlab which allows you to apply convolution in more dimensions.

edit retag flag offensive close merge delete

Comments

Hmm if I understand it correctly, then applying a convolution in 3D is just two times a 2D convolution? So wouldn't it just be combining 2D convolutions? Imagine a 3D x y z plane object. First convolve each XY layer with your 2D filter, then combine it back into ur 3D container and apply a 2D conv filter to each XZ plane. This should work and have same result.

StevenPuttemans gravatar imageStevenPuttemans ( 2016-01-08 07:52:34 -0500 )edit

@StevenPuttemans I will try your idea and see if that is the case. Thanks.

theodore gravatar imagetheodore ( 2016-01-08 10:35:04 -0500 )edit

I though that I got it but apparently not. Let's say that I have a depth image Z with the pixel values describing the distance. In Matlab what I saw people doing is n1 = conv3(Z, kernel);, where kernel is obviously the kernel used for the convolution. How this can be done with filter2d() I am not quite sure that it is possible.

theodore gravatar imagetheodore ( 2016-01-08 11:56:41 -0500 )edit

If you just have 3 dimensions, with the third being in the channels, you can do the 2D convolution on each channel, then call reshape(1, rowscols). The new matrix will be (Channel) columns by (rowscols) rows.

Then you call filter2d with the third dimension of your kernel as the X kernel parameter, and 1 as the Y kernel.

Then reshape back to normal.

Tetragramm gravatar imageTetragramm ( 2016-01-08 16:16:39 -0500 )edit

@Tetragramm would be easy to provide an example?

theodore gravatar imagetheodore ( 2016-01-08 18:32:54 -0500 )edit

Can you explain how do you manage 3d mat in opencv ? I have try this with a 3d histogram. May be you can look to this post.

LBerger gravatar imageLBerger ( 2016-01-09 11:36:08 -0500 )edit

actually I had something else in my mind, but nevermind let's keep the tread and the answer of @Tetragramm and your links @LBerger for future readers. Thanks for your response though ;-).

theodore gravatar imagetheodore ( 2016-01-10 06:25:30 -0500 )edit

maybe a small tip for @LBerger. I create my 3D mats using vector< vector<Mat> >, the inner vector is always a XY plane why the second vector represents the Z plane. Works perfectly fine.

StevenPuttemans gravatar imageStevenPuttemans ( 2016-01-11 02:24:01 -0500 )edit

@StevenPuttemans About convolution in 3d using separable filterI think you use filter2d for each plane (x,y direction) and for z direction a 1D convolution. For this 1D convolution which opencv function do you use with vector<vector <mat=""> > ?

LBerger gravatar imageLBerger ( 2016-01-13 03:44:44 -0500 )edit

No, it is again a 2D convolution in my mind, being of the XZ plane now. Then you get a 3D convolution in theory!

StevenPuttemans gravatar imageStevenPuttemans ( 2016-01-13 04:08:24 -0500 )edit

2 answers

Sort by ยป oldest newest most voted
1

answered 2016-01-09 10:54:14 -0500

Tetragramm gravatar image

You wanted an example, so here you go. This assumes the first two dimensions of the convolution have been done.

This does a BGR 2 Gray as a convolution between the channels. The center channel is properly gray, but the edges use reflected borders, so it does the math with gbg or grg, which of course doesn't come out correctly. Hopefully you can see how this would extend to images with a much higher number of channels.

image.create(100, 100, CV_8UC3);
image.setTo(cv::Scalar(50,100,200));
cv::cvtColor(image, gray, cv::COLOR_BGR2GRAY);

std::cout << image(cv::Rect(0, 0, 1, 1)) << "\n\n";
std::cout << gray(cv::Rect(0, 0, 1, 1)) << "\n\n";

cv::Mat thirdD, kernel;
kernel.create(1, 3, CV_32F);//3 is the Size of the kernel in the third dimension.
kernel.at<float>(0) = 0.114;//Blue
kernel.at<float>(1) = 0.587;//Green
kernel.at<float>(2) = 0.299;//Red
thirdD = image.reshape(1, image.rows*image.cols);
cv::filter2D(thirdD, output, -1, kernel, cv::Point(-1, -1), 0.0, cv::BORDER_REFLECT_101);//g|bgr|g
//The -1 says keep it in the same pixel format CV_8U.  You can choose different border types, but this is the default.
trueGray = output.reshape(3, image.rows);//3 is the number of channels you started with, the depth of the image in the third dimension.

std::cout << trueGray(cv::Rect(0, 0, 1, 1)) << "\n\n";
edit flag offensive delete link more

Comments

theodore gravatar imagetheodore ( 2016-01-10 06:25:59 -0500 )edit
1

answered 2016-01-10 19:14:47 -0500

updated 2016-01-10 19:16:45 -0500

Short answer: no, afaik there are no out of the box 3D convolution for arbitrary kernel in openCV.

Long answer: some convolution kernels are separable i.e. 2d convolution could be presented as a sequence of two 1D-convolution in one direction and then 1D in another direction (see sepFilter2D). The same for 3D -- some 3D kernels could be presented as a sequence of 1D or 2D convolution. For instance Gauss kernel is always separable. 1D convolution is much faster than 2D or 3D that's why one should always try to use separable kernels. If your particular filter is not separable you can try to approximate it as a superposition of separable filters (using SVD etc). But you always should think twice is it really so necessary to use not separable 3D filtering in your case.

edit flag offensive delete link more
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2016-01-08 05:20:57 -0500

Seen: 1,850 times

Last updated: Jan 10 '16