Ask Your Question
0

Wierd bug - cv::rotate corrupts images.

asked 2017-12-13 08:42:00 -0600

kmaster gravatar image

updated 2018-01-15 02:41:15 -0600

Hi,

I have been tracking this bug for quite some time now, so any suggestion would be welcome. I use OpenCV as part of the server, which uses OpenCV for image processing. The server is written in Golang and uses our custom binding to OpenCV.

One operation is to rotate images, and it works 99% of time correctly when using the same set of images.

Very very rarely the input image will fail (see attachments). There is nothing special about input images. Actually, the same image would go through rotation without corruption at any other time before suddenly failing.

On the server with 8 instances changing content every minute, it would fail once every few days.

I have written the unit test that repeats rotate and some other operations in multiple threads running in a loop. Test does not seam to catch any corruptions, but it does create heavy CPU and memory load.

Snippet showing how do we use cvTranspose() to implement image rotation. (It is a Go code using C bindings)

I have already posted the same question to slack, and I was given the recommendation to update to v 3.x.x. and use new 'rotate()' function.

I will probably try this, but the reason I post this here again is:

  • My concern is that there is either bug in OpenCV cvTranspose function (unlikely) or that there is the problem with the way we use OpenCV (more likely), and that in that case changing the function won't solve any problem. It is really hard for me right now to confirm that after the change I did solve the problem, because of bugs rare occurrence.
  • as a reference for a possible future bug report

Samples of corrupted images dumpede by 'cvTranspose()'. Images are gray but are actualy in RGBA format.

sample1

sample2

root@1c4b88d5e5b5:~/vss# uname -a

Linux 1c4b88d5e5b5 4.4.0-89-generic #112-Ubuntu SMP Mon Jul 31 19:38:41 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

root@1c4b88d5e5b5:~/vss# cat /etc/issue Ubuntu 16.04 LTS \n \l

root@1c4b88d5e5b5:~/vss# dpkg -l | grep opencv

ii libopencv-core2.4v5:amd64 2.4.9.1+dfsg-1.5ubuntu1 amd64 computer vision core library

ii libopencv-imgproc2.4v5:amd64 2.4.9.1+dfsg-1.5ubuntu1 amd64 computer vision Image Processing library

golang version 1.9.2

$ docker version # servers is running inside docker

Client:

Version: 17.09.0-ce

API version: 1.32

Go version: go1.8.3

Git commit: afdb6d4

Built: Tue Sep 26 22:42:18 2017

OS/Arch: linux/amd64

Server:

Version: 17.09.0-ce

API version: 1.32 (minimum version 1.12)

Go version: go1.8.3

Git commit: afdb6d4

Built: Tue Sep 26 22:40:56 2017

OS/Arch: linux/amd64

Experimental: false

edit retag flag offensive close merge delete

Comments

"The server is written in Golang and uses our custom binding to OpenCV."

yet, please don't use the no more maintained C-api, noone feels responsible to any errors coming from that. (they won't accept bug reports for this)

(and maybe something like this is more mature, than "rolling your own".)

berak gravatar imageberak ( 2017-12-13 09:31:11 -0600 )edit

@berak

https://github.com/gocv doesn't seem to be publicly available. "This organization has no public repositories."

At the time I was reviewing OpenCV bindings (2-3 years ago) there were not so many choices.

The choices that were available required copying pixel data between C "IpImage" and Go "image.Image".

For reasons of high-performance requirements, we wrote our bindings in the way so that data is shared. (notice how w we use CvCreateImageHeader() and cvSetData())

kmaster gravatar imagekmaster ( 2017-12-13 09:46:41 -0600 )edit
1

@berak I found working link for gocv ;D

kmaster gravatar imagekmaster ( 2017-12-13 09:50:35 -0600 )edit

yea, sorry, mistyped twice ;)

berak gravatar imageberak ( 2017-12-13 10:57:03 -0600 )edit

yea, sorry, seems that gocv does not handle your case (avoiding the copy on input)

is there any chance, your box is using ipp / opencl ? you might try to disable that for a moment, and see, how it goes (both are somewhat known to have problems with "in-place" things)

and again, while i do understand your problem, and i can't find any obvious flaw in the go-code, we'll have to close this for being "outdated and off-topic" sooner or later.

berak gravatar imageberak ( 2017-12-13 11:49:16 -0600 )edit

@berak Thank you for a suggestion. How do I know if I a box is using ipp/OpenCL? How do I disable it? Do I need to recompile?

What does in-place mean?

we do cvTranspose(in, out);

woud in-place mean cvTranspose(img, NULL); or cvTranspose(img, img) ? Are we dooing cvTranspose in place?

kmaster gravatar imagekmaster ( 2017-12-13 23:33:39 -0600 )edit

not the transpose, the flip, like here: C.cvFlip(unsafe.Pointer(cvOutImage), nil, C.int(CvFlipY))

you can set the env var OPENCV_OPENCL_DEVICE=disabled , to try without OCL at runtime (without recompiling anything)

berak gravatar imageberak ( 2017-12-14 01:11:46 -0600 )edit
1

@berak Glad to see I don't need to recompile ;D Yes I will need to confirm this with tests. And this will take some time

It is only that cvFlip() didn`t show up to be problematic in 2 dups I produced. Only cvTranspose().

Thank you for all the suggestions.

kmaster gravatar imagekmaster ( 2017-12-14 01:23:31 -0600 )edit
1

After some time I am back with some news:

What have I tried:

  • Upgraded to OpenCV 3.3
  • Re-compiled OpenCV without OpenCL and IPP support: -D WITH_IPP=OFF -D WITH_OPENCL=OFF -D WITH_OPENCLAMDFFT=OFF -D WITH_OPENCLAMDBLAS=OFF -D WITH_VA_INTEL=OFF
  • Replaced old C rotation code with modern C++ cv::rotate()

The problem persists after those changes. The average number should be close to one corruption to 50000 image rotations.

kmaster gravatar imagekmaster ( 2018-01-12 02:36:05 -0600 )edit

1 answer

Sort by ยป oldest newest most voted
4

answered 2018-01-22 06:55:09 -0600

kmaster gravatar image

As @break suspected, it was the problem with our wrapper. After code review and slight changes to unit tests to create appropriate conditions, we have confirmed that the problem was that Go GC could collect Go buffer while being used by OpenCV.

During code peer review I looked closely at passing image buffer (allocated in Go) to OpenCV. cgo Passing pointers

This gave me an idea:

We are looing two functions: - CvRotateOrtaghonal() - cvFromGoImageNoCopy()

Starting with CvRotateOrtaghonal() we try to rotate for 90*. First, we take image and convert it to OpenCV using cvFromGoImageNoCopy(). cvFromGoImageNoCopy() will take img. buffer and create OpenCV header for it. In the process, we call C.cvSetData() to store Go pointer inside C structure for later use. This is a direct violation of "cgo Passing pointers" rule:

C code may not keep a copy of a Go pointer after the call returns.

After this, we need OpenCV image, but inImage might be or might not be referenced anywhere in Go code anymore.

It certainly isn't referenced inside CvRotateOrtaghonal() anymore. For Go garbage collector this means, that memory is ready to be collected and can be reused by something else, while OpenCV is rotating this same buffer.

And even if garbage collector does not collect memory it can move it inside virtual address space at the whim, confusing C code in the process.

edit flag offensive delete link more

Comments

1

can't say, how happy am to see this resolved, finally !

(also, since i got some similar project combining go & opencv on the horizon ...)

berak gravatar imageberak ( 2018-01-22 07:52:52 -0600 )edit

@berak Yeas me too I am very happy. Thank you for your support and suggestions.

kmaster gravatar imagekmaster ( 2018-01-22 08:30:11 -0600 )edit
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2017-12-13 08:42:00 -0600

Seen: 130 times

Last updated: Jan 22