CascadeClassifier wont detect image using runAt that it has already found using dectectMultiScale

edit

I want to make a function that just runs the classifier once using runAt, for when you already know where the image should be. (E.g. if you have found a face but want to run several classifier over the face to decide if its happy or sad).

This was my attempt (I had to make a subclass since runAt is protected).

int RSCascadeClassifier::runOnceOnWholeImage(const cv::Mat& image,
                                             double & gypWeight)
{
    cv::Size size(image.cols, image.rows);

    // Need to set image first, see:
    // http://docs.opencv.org/modules/objdetect/doc/cascade_classification.html#featureevaluator-setimage
    bool success = featureEvaluator->setImage( image, size );

    if (!success)
    {
        CV_Assert( false );
        return 0;
    }

    // Once once over the whole image
    int result = runAt(this->featureEvaluator, cv::Point(0, 0), gypWeight);

    return result;
}

To test this out I got the result of a successful detectMultiScale, then cropped the image and sent it to my method:

faceLBPClassifier.detectMultiScale(fullImage,
                                    objects,
                                    1.1, // Scale factor
                                    2, // Min neighbours
                                    CV_HAAR_SCALE_IMAGE | 0, // Flags
                                    cv::Size( 80 , 120 ) // Min size
                                    );

if (objects.size() > 0)
{
    cv::Rect faceRect = objects[0];
    cv::Mat foundFace = fullImage(faceRect).clone();

    double weight;
    int result = faceLBPClassifier.runOnceOnWholeImage(foundFace, weight);

    NSLog(@"r:%d w:%f", result, weight);
}

The results were:

r:0 w:-2.135099
r:-2 w:-2.469498
r:0 w:-2.135099
r:-1 w:-3.106470
r:0 w:-2.135099  ...

I.e. no success (1) results.

This should always return success since the same classifier has already found the result in that the exact rectangle.

Does anyone know what I have done wrong?