# Meaning of "position of mapped points" in cv::cuda::meanShiftProc()

I am presently using and very satisfied with the implementation of the mean shift segmentation algorithm in the CUDA module. However, I cannot understand the meaning of the data contained in the dstsp output parameter in the cv::cuda::meanShiftProc() function. I can't find the necessary details in the documentation, and I can't guess the meaning of the "position of mapped points" from the literature, nor from the size and type of dstsp.

Can anyone offer me a hint? Does "position" mean some position in the color space, or in the image coordinates, or in the color histogram, or something completely different?

edit retag close merge delete

dstsp Destination image containing the position of mapped points. The size is the same as src size. The type is CV_16SC2 .

maybe looking at remapping helps to understand those. (for every pixel position, those contain the new, mapped position)

( 2018-12-21 05:57:28 -0500 )edit

Thanks for your comment. As I understand, cv::remap performs a (2D) geometrical transformation of the image, as opposed to meanShiftProc() which performs a (3D) transformation in the color space. Unless I am wrong on either of these points, the exact nature of the "position of mapped points" is still unclear.

Since the type of dstsp is the same as one of the types that can be used as the map for cv::remap, it sounds reasonable to assume that there is a connection between the two. Now the question is what kind of connection.

( 2018-12-21 08:03:40 -0500 )edit

Sort by » oldest newest most voted

The mean shift algorithm has been laid out in D. Comanciu and P. Meer, IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 24 (2002) pp. 603. The functions found in the cv::cuda module implement the variant of the algorithm with the joint domain representation of the pixel feature space. The joint domain representation means that each pixel is represented by five features, including three color coordinates and two spatial coordinates. As a result of the mean shift algorithm, each pixel is assigned a point in the 5D feature space corresponding to the location of the closest local maximum of pixel density distribution (a.k.a mode of the distribution).

In the context of image segmentation, the mean shift procedure in the joint feature space has at least two advantages over the variant that uses only the color domain:

• Small local features of the color distribution are not trumped by features that dominate the overall image.
• Spatially separate segments of similar average colors are recognized as separate segments.

The dstsp matrix maps each pixel of the original image to the (approximate) location of the mode it belongs to, projected onto the spatial part of the joint feature space. All pixels belonging to the same segment are mapped to approximately the same location, within the spatial tolerance given by the input parameter sp.

Similarly, the dstr matrix maps each pixel to the approximate location of its mode in the color space.

The dstsp and the dstr matrices are jointly used in the meanShiftSegmentation() function to cluster pixels into segments. Beyond this, the dstsp matrix need not have any special meaning. The use of the spatial coordinates in the mean shift algorithm simply prevents mixing of spatially separate modes.

more

Official site

GitHub

Wiki

Documentation