Ask Your Question

nan's in stitching_detailed example due to camera estimation?

asked 2013-04-30 15:37:40 -0600

ricor gravatar image

updated 2013-04-30 15:39:49 -0600

berak gravatar image

I've tried using the stitching_detailed example and while it works for a small set of images for more numerous cases it doesn't (e.g. >15). I find that in the camera estimation matrix log messages it is returning nan's. This of course means no panorama. I'm quite new to image processing so I'm struggling to understand what is going on here but I think it is to do with estimation of the focal length from the camera matrix.

I've looked for help in the docs but only found the opencvdoc API pages. I've therefore dug into the code in the stitching module and had a look at the offending bit in motion_estimators.cpp (code at the bottom).

However, can anyone help me in understanding with what is going on in that bit of code (below). In particular why is there a commented out bit (#if 0 to #endif) and what advantage would that give me? Also would this code be expected to be numerically unstable - probably most likely to be unstable in the call that reaches focalsFromHomography()?

Finally does anybody know of any explanatory docs to explain focalsFromHomography? It doesn't look like a factorisation of the camera matrix to me (which it possibly is in a different function where it starts talking about cholesky decomp). However, I'm very new to image processing so don't really know what I'm talking about here.

Many thanks for any tips on this

void HomographyBasedEstimator::estimate(const vector<ImageFeatures> &features, const vector<MatchesInfo> &pairwise_matches,
                                        vector<CameraParams> &cameras)
    LOGLN("Estimating rotations...");
    int64 t = getTickCount();

    const int num_images = static_cast<int>(features.size());

#if 0
    // Robustly estimate focal length from rotating cameras
    vector<Mat> Hs;
    for (int iter = 0; iter < 100; ++iter)
        int len = 2 + rand()%(pairwise_matches.size() - 1);
        vector<int> subset;
        selectRandomSubset(len, pairwise_matches.size(), subset);
        for (size_t i = 0; i < subset.size(); ++i)
            if (!pairwise_matches[subset[i]].H.empty())
        Mat_<double> K;
        if (Hs.size() >= 2)
            if (calibrateRotatingCamera(Hs, K))

    if (!is_focals_estimated_)
        // Estimate focal length and set it for all cameras
        vector<double> focals;
        estimateFocal(features, pairwise_matches, focals);
        cameras.assign(num_images, CameraParams());
        for (int i = 0; i < num_images; ++i)
            cameras[i].focal = focals[i];
        for (int i = 0; i < num_images; ++i)
            cameras[i].ppx -= 0.5 * features[i].img_size.width;
            cameras[i].ppy -= 0.5 * features[i].img_size.height;

    // Restore global motion
    Graph span_tree;
    vector<int> span_tree_centers;
    findMaxSpanningTree(num_images, pairwise_matches, span_tree, span_tree_centers);
    span_tree.walkBreadthFirst(span_tree_centers[0], CalcRotation(num_images, pairwise_matches, cameras));

    // As calculations were performed under assumption that p.p. is in image center
    for (int i = 0; i < num_images; ++i)
        cameras[i].ppx += 0.5 * features[i].img_size.width;
        cameras[i].ppy += 0.5 * features[i].img_size.height;

    LOGLN("Estimating rotations, time: " << ((getTickCount() - t) / getTickFrequency()) << " sec");
edit retag flag offensive close merge delete

1 answer

Sort by ยป oldest newest most voted

answered 2013-05-30 07:37:53 -0600

CamVis gravatar image

updated 2013-05-30 07:38:28 -0600

You can find detailed info in document "Construction of Panoramic Image Mosaics with Global and Local Alignment", Heung-Yeung Shum ([email protected]) and Richard Szeliski ([email protected])

The method "focals from homgraphy matrix" is implemented same as described in this document - page 17.

edit flag offensive delete link more

Question Tools


Asked: 2013-04-30 15:37:40 -0600

Seen: 1,159 times

Last updated: May 30 '13