Revision history [back]

Hello everyone!

I'm having some very weird issues with the Python bindings of OpenCV. I'm trying to use the dft function (amongst others) to compute correlations between long arrays of 1D data (think >=8192 samples).

If I set up a loop in my program and try to compute the same operations on the same input data several times (according to that loop) I get wildly different results. The correct result is the one that appears most times but still, sometimes it is not even 50% of the attempts (and whatever... it should be 100% same anyway!). It looked systematic at the beginning, but later I noticed that I could get varying degrees of success on different runs.

Trying to track the error, I debugged the code using Pydev and got to a line like this inside the loop (there are many other dfts in the code, this is the first one in the loop):

outputarray = cv2.dft(inputarray,flags=cv2.DFT_COMPLEX_OUTPUT)

Where inputarray is a constant array of real integers, one that has always the same contents (verified with the Pydev debugger). While the first iteration finishes perfectly fine, in the second iteration of the loop, outputarray has changed according to Pydev, despite using the same input and flags. Also, its max and min return "nan". It may return correct values in later iterations though.

I cannot post the source as is, but more or less it would be something like:

import numpy
import cv2

for i in nloops:

code
code

#functionA is set up to return always the same 1 channel ndarray
#for debugging of this problem, and it works as intended. (Thanks Pydev!)  
inputarray = functionA(...) 


#First cv2 operation, which already fails in the second iteration of the main loop:
outputarray = cv2.dft(inputarray,flags=cv2.DFT_COMPLEX_OUTPUT) 

#Then another internal loop comes that uses outputarray internally:
for j in rangevector:

    operations #More cv2.dft() and other openCV functions appear here

code
code
#end of function

All of this leads to me to thinking this is a bug within OpenCV (or its Python 2 bindings), but I struggle to believe such error flew under the radar during testing of several OpenCV releases (I tried with stock Ubuntu 14.04 and 15.04 64 bits).

Any useful advice? I will provide more information if needed.

Thank you very much!

Hello everyone!

I'm having some very weird issues with the Python bindings of OpenCV. I'm trying to use the dft function (amongst others) to compute correlations between long arrays of 1D data (think >=8192 samples).

If I set up a loop in my program and try to compute the same operations on the same input data several times (according to that loop) I get wildly different results. The correct result is the one that appears most times but still, sometimes it is not even 50% of the attempts (and whatever... it should be 100% same anyway!). It looked systematic at the beginning, but later I noticed that I could get varying degrees of success on different runs.

Trying to track the error, I debugged the code using Pydev and got to a line like this inside the loop (there are many other dfts in the code, this is the first one in the loop):

outputarray array_fix_S = cv2.dft(inputarray,flags=cv2.DFT_COMPLEX_OUTPUT) cv2.dft(array_fix,flags=cv2.DFT_COMPLEX_OUTPUT)

Where inputarrayarray_fix is a constant an array of real integers, values, one that has always the same contents (verified with the Pydev debugger). While the first iteration finishes perfectly fine, in the second iteration of the loop, outputarrayarray_fix_S has changed according to Pydev, despite using the same input and flags. Also, its max and min can return "nan". "nan" sometimes (even though I haven't seen that yet with the reproducing code I attach below). It may return correct values in later iterations though.

I cannot post the source as is, but more or less it would be something like:In my computer, the code below is enough to reproduce the issue. The numerical values are a little bit arbitrary. They are just close to what I really use in practice, not trying to exploit anything:

import numpy
cv2
import cv2
numpy as np

t = np.r_[0:16384]
#Random vector 1 to test dft repeatability. Will stay the same all the iterations.
array_fix = np.random.rand(16384) 

for i in nloops:

code
code

#functionA range (0,10):

    #Printing the randomly generated array_fix max and min
    #as an indicator (not proof, I admit) that the array
    #contents haven't changed. Still, taking a look
    # with Pydev debugger I confirmed no changes are detected.
    print(np.max(array_fix))
    print(np.min(array_fix))


    acum    = np.zeros((np.size(50),np.size(t)))

    ####AT LEAST THE OPERATION BELOW IS GIVING TROUBLE####
    array_fix_S = cv2.dft(array_fix,flags=cv2.DFT_COMPLEX_OUTPUT)

    #The 2 prints below can potentially change each iteration
    #despite coming from the same DFT every single time.
    print(np.max(array_fix_S))
    print(np.min(array_fix_S))


    for iteration in range(0, 50):

        #I and Q would normally be real/imaginary readings from
        #binary files that change each iteration in my code.
        #You can pull I and Q generation outside the loops like
        #array_fix and the problem in the previous prints still persists.
        I=np.random.rand(16384)
        Q=np.random.rand(16384)

        #Channel merging to get cv2-friendly complex array.
        xchan = [I,Q]
        X = cv2.merge(xchan)

        #If I comment the loop below (where the other dft/idft operations appear) the problem
        #in the previous prints is set up to return always the same 1 channel ndarray
#for debugging of this problem, and it works as intended. (Thanks Pydev!)  
inputarray GONE.
        for iteration2 in range(0, 50):
            interm = functionA(...) 


#First cv2 operation, which already fails in the second iteration of the main loop:
outputarray cv2.dft(X)
            sep = cv2.dft(inputarray,flags=cv2.DFT_COMPLEX_OUTPUT) 

#Then another internal loop comes that uses outputarray internally:
for j in rangevector:

    operations #More cv2.dft() and other openCV functions appear here

code
code
#end of function
cv2.split(cv2.idft(cv2.mulSpectrums((interm),array_fix_S,0,conjB=True),flags=cv2.DFT_COMPLEX_OUTPUT))

All of this leads to me to thinking this is a bug within OpenCV (or its Python 2 bindings), but I struggle to believe such error flew under the radar during testing of several OpenCV releases (I tried with stock Ubuntu 14.04 and 15.04 64 bits).

Any useful advice? I will provide more information if needed.

Bear in mind, the equivalent code using numpy.fft does work as intended. The algorithm is correct.

Thank you very much!

Edit: updated with proper working code to reproduce the issue.

Hello everyone!

I'm having some very weird issues with the Python bindings of OpenCV. I'm trying to use the dft function (amongst others) to compute correlations between long arrays of 1D data (think >=8192 samples).

If I set up a loop in my program and try to compute the same operations on the same input data several times (according to that loop) I get wildly different results. The correct result is the one that appears most times but still, sometimes it is not even 50% of the attempts (and whatever... it should be 100% same anyway!). It looked systematic at the beginning, but later I noticed that I could get varying degrees of success on different runs.

Trying to track the error, I debugged the code using Pydev and got to a line like this inside the loop (there are many other dfts in the code, this is the first one in the loop):

array_fix_S = cv2.dft(array_fix,flags=cv2.DFT_COMPLEX_OUTPUT)

Where array_fix is an array of real values, one that has always the same contents (verified with the Pydev debugger). While the first iteration finishes fine, in the second iteration of the loop, array_fix_S has changed according to Pydev, despite using the same input and flags. Also, its max and min can return "nan" sometimes (even though I haven't seen that yet with the reproducing code I attach below). It may return correct values in later iterations though.

In my computer, the code below is enough to reproduce the issue. The numerical values are a little bit arbitrary. They are just close to what I really use in practice, not trying to exploit anything:

import cv2
import numpy as np

t = np.r_[0:16384]
#Random vector 1 to test dft repeatability. Will stay the same all the iterations.
array_fix = np.random.rand(16384) 

for i in range (0,10):

    #Printing the randomly generated array_fix max and min
    #as an indicator (not proof, I admit) that the array
    #contents haven't changed. Still, taking a look
    # with Pydev debugger I confirmed no changes are detected.
    print(np.max(array_fix))
    print(np.min(array_fix))

print(np.min(array_fix)) 
    acum    = np.zeros((np.size(50),np.size(t)))

    ####AT LEAST THE OPERATION BELOW IS GIVING TROUBLE####
    array_fix_S = cv2.dft(array_fix,flags=cv2.DFT_COMPLEX_OUTPUT)

    #The 2 prints below can potentially change each iteration
    #despite coming from the same DFT every single time.
    print(np.max(array_fix_S))
    print(np.min(array_fix_S))
     for iteration in range(0, 50):

        #I and Q would normally be real/imaginary readings from
        #binary files that change each iteration in my code.
        #You can pull I and Q generation outside the loops like
        #array_fix and the problem in the previous prints still persists.
        I=np.random.rand(16384)
        Q=np.random.rand(16384)

        #Channel merging to get cv2-friendly complex array.
        xchan = [I,Q]
        X = cv2.merge(xchan)

        #If I comment the loop below (where the other dft/idft operations appear) the problem
        #in the previous prints is GONE.
        for iteration2 in range(0, 50):
            interm = cv2.dft(X)
            sep = cv2.split(cv2.idft(cv2.mulSpectrums((interm),array_fix_S,0,conjB=True),flags=cv2.DFT_COMPLEX_OUTPUT))

All of this leads to me to thinking this is a bug within OpenCV (or its Python 2 bindings), but I struggle to believe such error flew under the radar during testing of several OpenCV releases (I tried with stock Ubuntu 14.04 and 15.04 64 bits).

Any useful advice? I will provide more information if needed.

Bear in mind, the equivalent code using numpy.fft does work as intended. The algorithm is correct.

Thank you very much!

Edit: updated with proper working code to reproduce the issue.

Hello everyone!

everyone! I'm having some very weird issues with the Python bindings of OpenCV. I'm trying to use the dft function (amongst others) to compute correlations between long arrays of 1D data (think >=8192 samples).

If I set up a loop in my program and try to compute the same operations on the same input data several times (according to that loop) I get wildly different results. The correct result is the one that appears most times but sometimes it is not even 50% of the attempts (and whatever... it should be 100% same anyway!). It looked systematic at the beginning, but later I noticed that I could get varying degrees of success on different runs.

I debugged the code using Pydev and got to a line like this inside the loop (there are many other dfts in the code, this is the first one in the loop):

loop): array_fix_S = cv2.dft(array_fix,flags=cv2.DFT_COMPLEX_OUTPUT) cv2.dft(array_fix,flags=cv2.DFT_COMPLEX_OUTPUT)

Where array_fix is an array of real values, one that has always the same contents (verified with the Pydev debugger). While the The first iteration finishes fine, in the second iteration of the loop, fine but in later iterations array_fix_S has changed changes according to Pydev, despite using the same input and flags. Also, its max Max and min can return "nan" sometimes (even though I (I haven't seen that yet with the reproducing code I attach below). It may return correct values in later iterations though.

In my computer, the The following code below is enough to reproduce the issue. The numerical reproduces the problem. Numerical values are a little bit arbitrary. They are just close to what I really use in practice, practice/arbitrary, not trying to exploit anything:

import cv2
import numpy as np

t = np.r_[0:16384]
#Random vector 1 to test dft repeatability. Will stay the same all the iterations.
array_fix = np.random.rand(16384) 

for i in range (0,10):

    #Printing the randomly generated array_fix max and min
    #as an indicator (not proof, I admit) that the array
    #contents haven't changed. Still, taking a look
    # with Pydev debugger I confirmed no changes are detected.
    print(np.max(array_fix))
    print(np.min(array_fix)) 
    acum    = np.zeros((np.size(50),np.size(t)))

    ####AT LEAST THE OPERATION BELOW IS GIVING TROUBLE####
    array_fix_S = cv2.dft(array_fix,flags=cv2.DFT_COMPLEX_OUTPUT)

    #The 2 prints below can potentially change each iteration
    #despite coming from the same DFT every single time.
    print(np.max(array_fix_S))
    print(np.min(array_fix_S))
    for iteration in range(0, 50):

        #I and Q would normally be real/imaginary readings from
        #binary files that change each iteration in my code.
        #You can pull I and Q generation outside the loops like
        #array_fix and the problem in the previous prints still persists.
        I=np.random.rand(16384)
        Q=np.random.rand(16384)

        #Channel merging to get cv2-friendly complex array.
        xchan = [I,Q]
        X = cv2.merge(xchan)

        #If I comment the loop below (where the other dft/idft operations appear) the problem
        #in the previous prints is GONE.
        for iteration2 in range(0, 50):
            interm = cv2.dft(X)
            sep = cv2.split(cv2.idft(cv2.mulSpectrums((interm),array_fix_S,0,conjB=True),flags=cv2.DFT_COMPLEX_OUTPUT))

All of this This leads to me to thinking think this is a bug within OpenCV (or its Python 2 bindings), but I struggle to believe such error flew under the radar during testing of several OpenCV releases (I tried with stock Ubuntu 14.04 and 15.04 64 bits).

Any useful advice? I will provide more information if needed.

needed. Bear in mind, mind that the equivalent code using numpy.fft does work as intended. The algorithm is correct.

Thank you very much!

Edit: updated with proper working code to reproduce the issue.

Hello everyone! I'm having some very weird issues with the Python bindings of OpenCV. I'm trying to use the dft function (amongst others) to compute correlations between long arrays of 1D data (think >=8192 samples).

If I set up a loop in my program and try to compute the same operations on the same input data several times (according to that loop) I get wildly different results. The correct result is the one that appears most times but sometimes it is not even 50% of the attempts (and whatever... it should be 100% same anyway!). It looked systematic at the beginning, but later I noticed that I could get varying degrees of success on different runs.

I debugged the code using Pydev and got to a line like this inside the loop (there are many other dfts in the code, this is the first one in the loop): array_fix_S = cv2.dft(array_fix,flags=cv2.DFT_COMPLEX_OUTPUT)

Where array_fix is an array of real values, one that has always the same contents (verified with the Pydev debugger). The first iteration finishes fine but in later iterations array_fix_S changes according to Pydev, despite using the same input and flags. Max and min can return "nan" sometimes (I haven't seen that yet with the reproducing code I attach below). It may return correct values in later iterations though.

The following code reproduces the problem. Numerical values are just close to what I really use in practice/arbitrary, not trying to exploit anything:

import cv2
import numpy as np

t = np.r_[0:16384]
#Random vector 1 to test dft repeatability. Will stay the same all the iterations.
array_fix = np.random.rand(16384) 

for i in range (0,10):

    #Printing the randomly generated array_fix max and min
    #as min as an indicator (not proof, I admit) that the array
    #contents 
    # the array contents haven't changed. Still, taking Taking a look
    # look with the Pydev debugger I confirmed no changes 
    #changes are detected.
    print(np.max(array_fix))
    print(np.min(array_fix)) 
    acum    = np.zeros((np.size(50),np.size(t)))

    ####AT LEAST THE OPERATION BELOW IS GIVING TROUBLE####
    array_fix_S = cv2.dft(array_fix,flags=cv2.DFT_COMPLEX_OUTPUT)

    #The 2 prints below can potentially change each iteration
    #despite iteration despite coming from the same DFT every single time.
    print(np.max(array_fix_S))
    print(np.min(array_fix_S))
    for iteration in range(0, 50):

        #I and Q would normally be real/imaginary readings from
        #binary from binary files that change each change
        #each iteration in my code.
        #You code. You can pull I and & Q generation outside the loops like
        #array_fix and like array_fix
        #and the problem in the previous prints still persists.
        I=np.random.rand(16384)
        Q=np.random.rand(16384)

        #Channel merging to get cv2-friendly complex array.
        xchan = [I,Q]
        X = cv2.merge(xchan)

        #If I comment the loop below (where the other dft/idft operations appear) the problem
        #in the previous prints is GONE.
        for iteration2 in range(0, 50):
            interm = cv2.dft(X)
            sep = cv2.split(cv2.idft(cv2.mulSpectrums((interm),array_fix_S,0,conjB=True),flags=cv2.DFT_COMPLEX_OUTPUT))

This leads to me to think this is a bug within OpenCV (or its Python 2 bindings), but I struggle to believe such error flew under the radar during testing of several OpenCV releases (I tried with stock Ubuntu 14.04 and 15.04 64 bits).

Any useful advice? I will provide more information if needed. Bear in mind that the equivalent code using numpy.fft does work as intended. The algorithm is correct.

Thank you very much!

Edit: updated with proper working code to reproduce the issue.

Hello everyone! I'm having some very weird issues with the Python bindings of OpenCV. I'm trying to use the dft function (amongst others) to compute correlations between long arrays of 1D data (think >=8192 samples).

If I set up a loop in my program and try to compute the same operations on the same input data several times (according to that loop) I get wildly different results. The correct result is the one that appears most times but sometimes it is not even 50% of the attempts (and whatever... it should be 100% same anyway!). It looked systematic at the beginning, but later I noticed that I could get varying degrees of success on different runs.

I debugged the code using Pydev and got to a line like this inside the loop (there are many other dfts in the code, this is the first one in the loop): array_fix_S = cv2.dft(array_fix,flags=cv2.DFT_COMPLEX_OUTPUT)

Where array_fix is an array of real values, one that has always the same contents (verified with the Pydev debugger). The first iteration finishes fine but in later iterations array_fix_S changes according to Pydev, despite using the same input and flags. Max and min can return "nan" sometimes (I haven't seen that yet with the reproducing code I attach below). It may return correct values in later iterations though.

The following code reproduces the problem. Numerical values are just close to what I really use in practice/arbitrary, not trying to exploit anything:

import cv2
import numpy as np

t = np.r_[0:16384]
#Random vector 1 to test dft repeatability. Will stay the same all the iterations.
array_fix = np.random.rand(16384) 

for i in range (0,10):

    #Printing the randomly generated array_fix max and min as an indicator (not proof, I admit) that 
    # admit)
    #that the array contents haven't changed. Taking a look with the Pydev debugger I confirmed no 
    #changes confirmed
    #no changes are detected.
    print(np.max(array_fix))
    print(np.min(array_fix)) 
    acum    = np.zeros((np.size(50),np.size(t)))

    ####AT LEAST THE OPERATION BELOW IS GIVING TROUBLE####
    array_fix_S = cv2.dft(array_fix,flags=cv2.DFT_COMPLEX_OUTPUT)

    #The 2 prints below can change each iteration despite coming from the same DFT every time.
    print(np.max(array_fix_S))
    print(np.min(array_fix_S))
    for iteration in range(0, 50):

        #I and Q would normally be real/imaginary readings from binary files that change
        #each iteration in my code. You can pull I & Q generation outside the loops like array_fix
        #and like
        #array_fix and the problem still persists.
        I=np.random.rand(16384)
        Q=np.random.rand(16384)

        #Channel merging to get cv2-friendly complex array.
        xchan = [I,Q]
        X = cv2.merge(xchan)

        #If I comment the loop below (where the other dft/idft operations appear) the problem
        #in the previous prints is GONE.
        for iteration2 in range(0, 50):
            interm = cv2.dft(X)
            sep = cv2.split(cv2.idft(cv2.mulSpectrums((interm),array_fix_S,0,conjB=True),flags=cv2.DFT_COMPLEX_OUTPUT))

This leads to me to think this is a bug within OpenCV (or its Python 2 bindings), but I struggle to believe such error flew under the radar during testing of several OpenCV releases (I tried with stock Ubuntu 14.04 and 15.04 64 bits).

Any useful advice? I will provide more information if needed. Bear in mind that the equivalent code using numpy.fft does work as intended. The algorithm is correct.

Thank you very much!

Edit: updated with proper working code to reproduce the issue.

Hello everyone! I'm having some very weird issues with the Python bindings of OpenCV. I'm trying to use the dft function (amongst others) to compute correlations operations between long arrays of 1D data (think >=8192 samples).

If I set The issue is, when setting up a loop in my program and try to compute certain combination of loops (see code reproducing the problem below), a dft that is supposed to give always the same operations on the same input answer can output different data several times (according to that loop) I get wildly in different results. The correct result is the one that appears most times but sometimes it is not even 50% of the attempts (and whatever... it should be 100% same anyway!). It looked systematic at the beginning, but later I noticed that I could get varying degrees of success on different runs.

I debugged the code using Pydev and got to a line like this inside the loop (there are many other dfts in the code, this is the first one in the loop): array_fix_S = cv2.dft(array_fix,flags=cv2.DFT_COMPLEX_OUTPUT)

Where array_fix is an array of real values, one that has always the same contents (verified with the Pydev debugger). The first iteration finishes fine but in later iterations (array_fix_S changes according to Pydev, despite using the same input and flags. Max and min can return "nan" sometimes (I haven't seen that yet with the reproducing code I attach below). It may return correct values in later iterations though.

The following code reproduces the problem. ).

Numerical values are just close to what I really use in practice/arbitrary, not trying to exploit anything:

import cv2
import numpy as np

t = np.r_[0:16384]
#Random vector 1 to test dft repeatability. Will stay the same all the iterations.
array_fix = np.random.rand(16384) 

for i in range (0,10):

    #Printing the randomly generated array_fix max and min as an indicator (not proof, I admit)
    #that the array contents haven't changed. Taking a look with the Pydev debugger I confirmed
    #no changes are detected.
#This print is always the same, which is OK.
    print(np.max(array_fix))
    print(np.min(array_fix)) 
    acum    = np.zeros((np.size(50),np.size(t)))

    ####AT LEAST THE OPERATION BELOW IS GIVING TROUBLE####
    array_fix_S = cv2.dft(array_fix,flags=cv2.DFT_COMPLEX_OUTPUT)

    #The 2 prints below can change each iteration despite coming from the same DFT every time.
    #Pydev's debugger shows that the arrays do change between iterations when it happens.
    print(np.max(array_fix_S))
    print(np.min(array_fix_S))
    for iteration in range(0, 50):

        #I and Q would normally be real/imaginary readings from binary files that change
        #each iteration in my code. You can pull I & Q generation outside the loops like
        #array_fix and the problem in array_fix_S still persists.
        I=np.random.rand(16384)
        Q=np.random.rand(16384)

        #Channel merging to get cv2-friendly complex array.
        xchan = [I,Q]
        X = cv2.merge(xchan)

        #If I comment the loop below (where the other dft/idft operations appear) the problem
        #in the previous prints is GONE.
        for iteration2 in range(0, 50):
            interm = cv2.dft(X)
            sep = cv2.split(cv2.idft(cv2.mulSpectrums((interm),array_fix_S,0,conjB=True),flags=cv2.DFT_COMPLEX_OUTPUT))

This leads to me Bear in mind that the equivalent code using numpy.fft does work as intended. The algorithm in principle is correct. I'm starting to think this is a bug within an OpenCV (or its Python 2 bindings), but bug.

I struggle to believe such error flew under the radar during testing of several OpenCV releases (I tried with stock Ubuntu 14.04 and 15.04 64 bits).bits.

Any useful advice? I will provide more information if needed. Bear in mind that the equivalent code using numpy.fft does work as intended. The algorithm is correct.needed.

Thank you very much!