Cloud extraction from sky image

edit

I'm trying to achieve clouds extraction from a sky image(which I get from camera/video) with my limited OpenCV knowledge.

In the code example below, I'm splitting the image to it's channels such as BGR, and I'm getting grayscale image by blue-red channels difference. Then I put threshold on this grayscale image to create cloud area mask(in fact, it gives sky areas and I invert it). After these steps, finally I put the mask on the actual image which is show the real cloud areas according to the threshold limit. It works as expected for whitish/grayish clouds but it doesn't for the bluish ones. After some additional checks about sky characteristics such as NBRR(Normalised Blue-Red Ratio) and saturation, I got better results but still not best in my opinion due to threshold values need to be changed image to image.

const int BINARY_TH = 35;
const double SAT_FIX_TH = 0.3;
const double NBRR_FIX_TH = 0.4;

cv::Mat ch[3], img_gray, img_element, img_brdiff;
img_element = getStructuringElement(MORPH_RECT, Size(6, 6), Point(6, 6));
img_brdiff = cv::Mat::zeros(img_frame.size(), CV_8UC3);
cv::split(img_frame, ch);
cv::absdiff(ch[2], ch[0], img_brdiff);
cv::threshold(img_brdiff, img_gray, BINARY_TH, 255, CV_THRESH_BINARY_INV);      
cv::morphologyEx(img_gray, img_gray, cv::MORPH_CLOSE, img_element);

double r, b, g, nbrr, sat;
for(int y = 0; y < img_frame.rows; y++)
{
    for(int x = 0; x < img_frame.cols; x++)
    {
        b = img_frame.at<Vec3b>(y,x)[0];
        g = img_frame.at<Vec3b>(y,x)[1];
        r = img_frame.at<Vec3b>(y,x)[2];

        nbrr = (b - r) / (b + r);
        sat = 1.0 - (std::min(b, std::min(g, r)) / std::max(b, std::max(g, r)));

        if( nbrr < NBRR_FIX_TH &&
            sat < SAT_FIX_TH
            )
            img_gray.at<uchar>(y, x) = (uchar)255;
    }
}

cv::medianBlur(img_gray, img_gray, 3);

img_frame.copyTo(img_final, img_gray);
cv::imshow("in", img_frame);
cv::imshow("out", img_final);

Sample output :

Any idea about how to improve the technique? Also, in step two, I need to calculate move direction for each cloud.