Difference between output of grabCut between runs
Hello everyone,
When running grabCut on an the same input in the loop every time I will get the different output, but if I restart the program, the output, even though different within the loop, will match the result of corresponding iteration from the previous run. I know, that grabCut will use some singletons so that the instance will be there through all the runs until the program will be terminated, but still, why do the separate runs of the algorithm interfere with each other (somehow) and what to do to get around that?
I checked it with the unit test. If on one run I will save the results to disk like this:
TEST(PreprocessingTests, CutAndFilterSave)
{
std::string output_directory = "cutAndFilterPreviousRun";
std::experimental::filesystem::create_directory(output_directory);
cv::Mat hidden_mouth_image = cv::imread("hidden_mouth.png", cv::IMREAD_GRAYSCALE);
EXPECT_TRUE(hidden_mouth_image.data);
cv::Mat whole_face_image = cv::imread("whole_face.png", cv::IMREAD_GRAYSCALE);
EXPECT_TRUE(whole_face_image.data);
std::vector<cv::Mat> images_to_save;
images_to_save.emplace_back(hidden_mouth_image);
images_to_save.emplace_back(whole_face_image);
processImagesForComparison(images_to_save);
cv::imwrite(output_directory + "/hidden_mouth_filtered.png", images_to_save.at(0));
cv::imwrite(output_directory + "/whole_face_filtered.png", images_to_save.at(1));
}
And on second run I will run this function
TEST(PreprocessingTests, CutAndFilterRead)
{
std::string output_directory = "cutAndFilterPreviousRun";
if (std::experimental::filesystem::exists(output_directory))
{
cv::Mat hidden_mouth_image_filtered = cv::imread(output_directory + "/hidden_mouth_filtered.png", cv::IMREAD_GRAYSCALE);
EXPECT_TRUE(hidden_mouth_image_filtered.data);
cv::Mat whole_face_image_filtered = cv::imread(output_directory + "/whole_face_filtered.png", cv::IMREAD_GRAYSCALE);
EXPECT_TRUE(whole_face_image_filtered.data);
std::vector<cv::Mat> previous_run_filtered_images;
previous_run_filtered_images.emplace_back(hidden_mouth_image_filtered);
previous_run_filtered_images.emplace_back(whole_face_image_filtered);
cv::Mat hidden_mouth_image = cv::imread("hidden_mouth.png", cv::IMREAD_GRAYSCALE);
EXPECT_TRUE(hidden_mouth_image.data);
cv::Mat whole_face_image = cv::imread("whole_face.png", cv::IMREAD_GRAYSCALE);
EXPECT_TRUE(whole_face_image.data);
std::vector<cv::Mat> test_images;
test_images.emplace_back(hidden_mouth_image);
test_images.emplace_back(whole_face_image);
processImagesForComparison(test_images);
EXPECT_TRUE(areTheSame(test_images, previous_run_filtered_images));
}
}
Here areTheSame
will return true. But if I run it in the loop:
TEST(PreprocessingTests, CutAndFilterTest)
{
cv::Mat hidden_mouth_image = cv::imread("hidden_mouth.png", cv::IMREAD_GRAYSCALE);
EXPECT_TRUE(hidden_mouth_image.data);
cv::Mat whole_face_image = cv::imread("whole_face.png", cv::IMREAD_GRAYSCALE);
EXPECT_TRUE(whole_face_image.data);
std::vector<cv::Mat> original_images;
for (int i=0; i < 1; i++)
{
original_images.emplace_back(hidden_mouth_image);
original_images.emplace_back(whole_face_image);
}
auto copied_original_images = cloneMatVector(original_images);
std::vector<cv::Mat> previous_filtered_images;
for (int run = 0; run < 2; run++)
{
std::vector<cv::Mat> run_images = cloneMatVector(original_images);
processImagesForComparison(run_images);
if (!previous_filtered_images.empty())
EXPECT_TRUE(areTheSame(run_images, previous_filtered_images));
previous_filtered_images = cloneMatVector(run_images);
}
EXPECT_TRUE(areTheSame(original_images, copied_original_images));
}
then areTheSame
will fail... In processImagesForComparison
my grabCut implementation will be run on each vector element:
cv::Mat grabCutSegmentation(const cv::Mat& input)
{
cv::Mat bgModel, fgModel;
cv::Mat mask(input.rows, input.cols, CV_8U);
// let's set all of them to possible background first
mask.setTo(cv::GC_PR_BGD);
// cut out a small area in the middle of the image
int m_rows = 0.75 * input.rows;
int m_cols = 0.6 * input.cols;
// the region of interest
cv::Mat fg_seed = mask(cv::Range(input.rows/2 - m_rows/2, input.rows/2 + m_rows/2),
cv::Range(input.cols/2 - m_cols/2, input.cols/2 + m_cols/2));
// mark it as foreground
fg_seed.setTo(cv::GC_FGD);
// select last 3 rows of the image as background
cv::Mat1b bg_seed = mask(cv::Range(mask ...
oh, wait, that was about the threadpool memory used in parallel_for_, not about the grabcut algorithm !
sorry, but it means you have to dive into the src code now. e.g. the GMM part uses kmeans with KMEANS_PP_CENTERS. you'll have to investigate, how random is used there, if or how the seed is initialized there, etc.
I don't know... Was it? I am still fighting with the same issue as before, grabCut works nice here for segmentation, wanted to find the source of the problem before deciding to switch to different algorithm, maybe we can solve it somehow.
Thank you berak, it really seems to be some issue with randomization here, thank you! I will start with kmeans as you recommended.
But I guess it will not help me in the end, the result is going to be random and algorithm non deterministic. I will have to run it a couple of times to get the mean during result verification. Still I will try to find where the seed is set and make a push request. Thank you for your help berak!
kmeans.cpp:276
RNG& rng = theRNG();
so this means: the global random generator is used. it is seeded with a fixed number on startup of the program.
yea, it will do the same thing as the last time
Got it, thank you. It wouldn't be difficult to change to my generator with different seed in those functions, but I don't know how far it is helpful. Good to know that there are no memory leaks or similar ;)
oh, don't try to change it, it would making tests impossible.