Ask Your Question
0

Cuda ORB keypoints

asked 2020-01-19 08:04:36 -0500

MicTie gravatar image

I have a little experience with OpenCV (Python) and just received a Jetson Nano and want to test OpenCV with CUDA. So I tried an ORB-example but am stuck with converting GPU-Keypoints to CPU-Keypoints, but I am unable to get any result.


def orb_with_cuda():
        MAX_FEATURES = 100
        GOOD_MATCH_PERCENT = 0.15
        # load images into numpy
        npMat1 = get_sample("im1.jpg")
        npMat2 = get_sample("im2.jpg")        
        # upload into Cuda
        cuMat1 = cv2.cuda_GpuMat()
        cuMat2 = cv2.cuda_GpuMat()
        cuMat1.upload(npMat1)
        cuMat2.upload(npMat2)        
        #convert to Gray
        cuMat1g = cv2.cuda.cvtColor(cuMat1, cv2.COLOR_RGB2GRAY)
        cuMat2g = cv2.cuda.cvtColor(cuMat2, cv2.COLOR_RGB2GRAY)

        #ORB
        corb = cv2.cuda_ORB.create(MAX_FEATURES)
        _kps1, _descs1 = corb.detectAndComputeAsync(cuMat1g, None)
        _kps2, _descs2 = corb.detectAndComputeAsync(cuMat2g, None)

        #convert Keypoints to CPU
        kps1 = [cv2.KeyPoint() for i in range(MAX_FEATURES)]
        kps2 = [cv2.KeyPoint() for i in range(MAX_FEATURES)]
        corb.convert(_kps1, kps1)
        corb.convert(_kps2, kps2)      

        #BruteForce Matching
        cbf = cv2.cuda_DescriptorMatcher.createBFMatcher(cv2.NORM_HAMMING)
        cmatches = cbf.match(_descs1, _descs2) 
        # Sort matches by score
        cmatches.sort(key=lambda x: x.distance, reverse=False)
        # Remove not so good matches
        numGoodMatches = int(len(cmatches) * GOOD_MATCH_PERCENT)
        cmatches = cmatches[:numGoodMatches]  
        # Draw top matches
        imMatches = cv2.drawMatches(npMat1, kps1, npMat2, kps2, cmatches, None)
        cv2.imwrite("gpu_matches.jpg", imMatches)
        return()

The code above is running w/o any errors, but it seems the CPU-keypoints are all empty (the created images does not show any matching lines). The equivalent Non-CUDA code is running fine.

Any idea what I am doing wrong? I tried several conversion methods, this is the 'best' I can find. Thanks for your help!

(Using OpenCV4.1.2 on Jetson Nano with Python3.6.9)

edit retag flag offensive close merge delete

Comments

Change this:

# upload into Cuda
cuMat1 = cv2.cuda_GpuMat()
cuMat2 = cv2.cuda_GpuMat()
 cuMat1.upload(npMat1)
 cuMat2.upload(npMat2)        
 #convert to Gray
 cuMat1g = cv2.cuda.cvtColor(cuMat1, cv2.COLOR_RGB2GRAY)
 cuMat2g = cv2.cuda.cvtColor(cuMat2, cv2.COLOR_RGB2GRAY)

to:

cuMat1 = cv2.cuda_GpuMat()
cuMat2 = cv2.cuda_GpuMat()

cuMat1g = cv2.cuda.cvtColor(npMat1, cv2.COLOR_RGB2GRAY)
cuMat2g = cv2.cuda.cvtColor(npMat1, cv2.COLOR_RGB2GRAY)
cuMat1.upload(cuMat1g)
cuMat2.upload(cuMat2g)
supra56 gravatar imagesupra56 ( 2020-01-19 10:50:25 -0500 )edit

Have you tried inspecting the result of

_kps1.download()
cudawarped gravatar imagecudawarped ( 2020-01-19 12:17:18 -0500 )edit

@supra56: Thanks for your help! I changed the lines to


    cuMat1 = cv2.cuda_GpuMat()
    cuMat2 = cv2.cuda_GpuMat()
    cuMat1g = cv2.cvtColor(npMat1, cv2.COLOR_RGB2GRAY)
    cuMat2g = cv2.cvtColor(npMat2, cv2.COLOR_RGB2GRAY)
    cuMat1.upload(cuMat1g)
    cuMat2.upload(cuMat2g)

The overall result stays the same.

MicTie gravatar imageMicTie ( 2020-01-20 01:44:11 -0500 )edit

@cudawarped: Thanks for your help!

_kps1.download() shows a <class 'numpy.ndarray'=""> with


[[2.54100000e+03 4.51000000e+02 2.27500000e+03 2.21700000e+03
  2.31100000e+03 2.02300000e+03 1.73000000e+03 2.32700000e+03
  4.10000000e+02 2.27400000e+03 3.12000000e+02 2.60800000e+03
  2.27000000e+03 2.28900000e+03 2.29300000e+03 2.23300000e+03
  1.05000000e+03 2.23800000e+03 1.97700000e+03 2.27300000e+03
  2.25500000e+03 2.01500000e+03 1.07040002e+03 2.53920020e+03
  2.30400000e+03 9.21600037e+02 2.54040015e+03 2.63400000e+03
  ......

If I use _kps1 and _kps2 in "imMatches = cv2.drawMatches(npMat1, _kps1, npMat2, _kps2, cmatches, None), I receive an Error Message SystemError: <built-in function="" drawmatches=""> returned NULL without setting an error

MicTie gravatar imageMicTie ( 2020-01-20 02:01:44 -0500 )edit

Do you mean

imMatches = cv2.drawMatches(npMat1, _kps1.download(), npMat2, _kps2.download(), cmatches, None)

is cmatches a GpuMat() or numpy array?

The functions in the CUDA namespace (cv2.cuda cv2.cuda_) require the input/output arrays to be GpuMat(), if you want to use them with functions which are not in the CUDA namepsace (cv.) then you will need to download() them to the host first. This allows you to control when the costly memory transfers to/from the device/host take place.

cudawarped gravatar imagecudawarped ( 2020-01-20 03:37:29 -0500 )edit

cmatches is <class 'list'=""> of <class 'cv2.dmatch'="">.

You, are right I did some nonsense in my earlier reply.

If I use imMatches = cv2.drawMatches(npMat1, _kps1.download(), npMat2, _kps2.download(), cmatches, None), I receive a TypeError: Expected cv::KeyPoint for argument '%s'. My initial attempt (1st post) used the type cv:KeyPoint

MicTie gravatar imageMicTie ( 2020-01-20 03:52:14 -0500 )edit

Does this help imMatches = cv2.drawMatches(npMat1, kps1, npMat2, kps2, cmatches[10], None)? Use any number from 1 to 10 using your owned code. If not try both mine and@ cudawraped together or one at a time.

supra56 gravatar imagesupra56 ( 2020-01-20 06:34:59 -0500 )edit

@supra56: using

cuMat1 = cv2.cuda_GpuMat() 
cuMat2 = cv2.cuda_GpuMat() 
cuMat1g = cv2.cvtColor(npMat1, cv2.COLOR_RGB2GRAY) 
cuMat2g = cv2.cvtColor(npMat2, cv2.COLOR_RGB2GRAY) 
cuMat1.upload(cuMat1g) cuMat2.upload(cuMat2g) 
...... 
imMatches = cv2.drawMatches(npMat1, kps1, npMat2, kps2, cmatches[10], None)

returns SystemError: <built-in function="" drawmatches=""> returned NULL without setting an error. Same for other numbers .

`imMatches = cv2.drawMatches(npMat1, kps1, npMat2, kps2, cmatches[:10], None)` produces no error, but still no matches

I am trying various code-permutations, but no luck ....

MicTie gravatar imageMicTie ( 2020-01-20 07:11:07 -0500 )edit

Have you tried passing KeyPoints() instead of numpy arrays?

npKps1 = _kps1.download()
kps1 = [cv.KeyPoint(*npKps1[:,i]) for i in range(npKps1.shape[1])]
npKps2 = _kps2.download()
kps2 = [cv.KeyPoint(*npKps2[:,i]) for i in range(npKps2.shape[1])]
cudawarped gravatar imagecudawarped ( 2020-01-20 09:02:14 -0500 )edit

Thanks, this seems to be working! I have to wait for a couple of hours before I can show the answer

MicTie gravatar imageMicTie ( 2020-01-20 23:54:25 -0500 )edit

1 answer

Sort by ยป oldest newest most voted
0

answered 2020-01-20 12:09:10 -0500

Hi, I have had chance to look at this now. I think the real problem is that the python wrapper for corb.convert() does not allow the host KeyPoints to be an output argument. I would suggest adding CV_OUT as shown below

CV_WRAP virtual void convert(InputArray gpu_keypoints,
                     CV_OUT std::vector<KeyPoint>& keypoints) = 0;

to line 417 of cudafeatures2d.hpp. Hopefully then drawing matches should be as simple as

imMatches = cv.drawMatches(npMat1, corb.convert(_kps1), npMat2, corb.convert(_kps2), cmatches, None)
edit flag offensive delete link more

Comments

Thanks for your help! Too bad the solution is not working: The line imMatches = cv2.drawMatches(npMat1, corb.convert(_kps1), npMat2, corb.convert(_kps2), cmatches, None) draws an error: TypeError: Required argument 'keypoints' (pos 2) not found

MicTie gravatar imageMicTie ( 2020-01-20 19:20:57 -0500 )edit

It's working on github, are you sure you re-compiled, and copied the new python bindings to your site-packages folder?

cudawarped gravatar imagecudawarped ( 2020-01-21 02:42:39 -0500 )edit

Sorry, did not know I have to recompile ... Do I have to recompile the entire OpenCV, or is there a faster way? (using Linux 18.04)

MicTie gravatar imageMicTie ( 2020-01-21 03:14:12 -0500 )edit
1

If you start the build again (don't re-build) then only the required elements should be re-built. On windows with ninja this takes less than 30 seconds.

cudawarped gravatar imagecudawarped ( 2020-01-21 03:32:21 -0500 )edit
1

I re-compiled the .hpp and everything is working! Thank you so much for your help!

MicTie gravatar imageMicTie ( 2020-01-21 07:49:58 -0500 )edit
Login/Signup to Answer

Question Tools

2 followers

Stats

Asked: 2020-01-19 08:04:36 -0500

Seen: 526 times

Last updated: Jan 20