Implementation of Absolute Differences Stereo Matching Algorithm

asked 2015-05-30 22:18:13 -0500

renzbagaporo gravatar image

Good day!

I am trying to learn how to manually implement stereo matching algorithms. I'm basically starting with the most basic of them all - Absolute Difference.

I found some slides online that describe how to do it. Basically, from what I understand, I should calculate the difference between the pixels in my left image and the same pixel in the right image "shifted" by a certain distance/disparity. Then among these disparities, I select the minimum, which makes sense to me since the pixel with the lowest disparity means that it is most likely the same pixel in the left image.

I've coded it in OpenCV. Here is the code:

include <opencv2\opencv.hpp>
#include "support.h"

int main(){
    cv::Mat left_img = cv::imread("..\\images\\middlebury\\tsu_left.png", cv::IMREAD_GRAYSCALE);
    cv::Mat right_img = cv::imread("..\\images\\middlebury\\tsu_right.png", cv::IMREAD_GRAYSCALE);

    int disparity_max = 64;

    std::vector<cv::Mat> cost_maps;

    for (int disparity = 0; disparity < disparity_max; disparity++){
        cv::Mat cost_map = cv::Mat(left_img.size().height, left_img.size().width, CV_8UC1);
        for (int row = 0; row < left_img.size().height; row++){
            for (int col = 0; col < left_img.size().width; col++){
                int col_disp = (col - disparity < 0) ? 0 : col - disparity;
                            //Absolute Difference
                cost_map.at<uchar>(row, col) = abs(left_img.at<uchar>(row, col) - right_img.at<uchar>(row, col_disp));
            }
        }
        cost_maps.push_back(cost_map);
        show(cost_map);
    }


    //Pick the smallest disparity
    cv::Mat final_cost_map = cv::Mat(left_img.size().height, left_img.size().width, CV_8UC1);
    for (int row = 0; row < final_cost_map.size().height; row++){
        for (int col = 0; col < final_cost_map.size().width; col++){

            int min = 65536;
            for (int disparity = 0; disparity < disparity_max; disparity++){
                min = (cost_maps[disparity].at<uchar>(row, col) < min ? cost_maps[disparity].at<uchar>(row, col) : min);
            }

            final_cost_map.at<uchar>(row, col) = min;

        }
    }

    //Show the final cost map, multiplied by 32 to show pixels with low value
    show(final_cost_map * 32);
}

Take note that I haven't implemented the variant where the differences in a certain window is summed, leading to Sum of Absolute Differences. I am only taking the difference per pixel, per disparity. The lecture slides I found online says that it should look like this (rightmost image):

https://dl.dropboxusercontent.com/u/9...

However, the result from the code above yield something like this:

https://www.dropbox.com/s/7ns7h6tqycj...

I can't figure out why the outputs are so different. Is there some trivial step that I am missing, or is my understanding of how the algorithm works wrong? I am also using the tsukuba image.

edit retag flag offensive close merge delete

Comments

You need to sum the differences otherwise the error level will be so high you are unlikely to see a useful output. The left/right camera images will have pixels for the same objects with slightly different values due to lighting, camera noise, im perfect calibration, so the whole process is prone to error.

Boogaloo gravatar imageBoogaloo ( 2016-03-06 05:38:55 -0500 )edit