Ask Your Question
1

FLANN Index in Python - Training Fails with Segfault

asked 2014-10-16 22:17:43 -0500

Christopher gravatar image

updated 2014-10-17 14:53:25 -0500

Dear Internet,

I'm trying to add a number of images to a FLANN index (thousands in reality, once I have this working) and then find the closest match in the index to a query image. But it segfaults. :(

Bare essentials code:

import numpy as np
import cv2
import sys


FLANN_INDEX_LSH = 6


img1 = cv2.imread(sys.argv[1],0)


brisk = cv2.BRISK()


kp1, des1 = brisk.detectAndCompute(img1,None)


index_params= dict(algorithm = FLANN_INDEX_LSH,
                   table_number = 6, # 12
                   key_size = 12,     # 20
                   multi_probe_level = 1) #2
search_params = dict(checks=50)   # or pass empty dictionary

flann = cv2.FlannBasedMatcher(index_params,search_params)

for filename in sys.argv[2:]:
    img2 = cv2.imread(filename, 0)
    print "Detecting and computing {0}".format(filename)
    kp2, des2 = brisk.detectAndCompute(img2,None)
    print "Adding..."
    flann.add(des2)


print len(flann.getTrainDescriptors()) #verify that it actually took the descriptors in

print "Training..."
flann.train()                    

print "Matching..."
matches = flann.knnMatch(des1,k=2)

(I've tried this both with 2.4.9 and a pull of 3.0 alpha as from the day I'm posting this, both had same result.)

Problem: Here's the output where it fails:

Training...

Segmentation fault (core dumped)

So it dies at training, or at match() if I skip training.

What has me stuck:

  1. I can't find this functionality in the docs as regards Python. Am I in the wrong places?
    http://docs.opencv.org/trunk/modules/flann/doc/flann_fast_approximate_nearest_neighbor_search.html
    http://docs.opencv.org/trunk/modules/features2d/doc/common_interfaces_of_descriptor_matchers.html

  2. For lack of docs, I'm trying to emulate in Python what I see in matching_to_many_images.cpp from the samples since that's the closest thing to what I'm trying to do. I think. So that's where I'm getting what I have so far.

Now what? Kind of lost. Sorry if this is actually clearly documented and I just couldn't find it.

Thanks!

edit retag flag offensive close merge delete

Comments

It could be a problem with LSH. Try with KDTREE algo param in your FLANN params, and you'll go farther. Also, you should see this: http://stackoverflow.com/a/13654489

EDIT: I think it's a bug in opencv. Training a flann based matcher results in segmentation fault (some pointer trying to access a memory location something that doesn't exist, probably). Try with simply matching two images one on one, and it works.

So, I guess the fault is somewhere in OpenCV. I used the features2d interfaces for creating detectors and descriptors extractors.

bad_keypoints gravatar imagebad_keypoints ( 2014-10-17 07:56:35 -0500 )edit

Also, I think you should try out BRISK from here, see if that works, with your 3.0 build: https://github.com/Itseez/opencv/blob/master/samples/python2/find_obj.py

bad_keypoints gravatar imagebad_keypoints ( 2014-10-17 08:54:47 -0500 )edit

To answer your questions:

Matching one to one worked just fine. In fact, I tested that with find_obj.py!

But then I came upon something else. http://stackoverflow.com/questions/25781782/making-flann-matcher-editable-and-savable-to-disk ...which passed descriptors inside a list to add().

So I made a switch that no longer causes the segfault:

This:<br> flann.add([des2]) <br> instead of: <Br> flann.add(des2) <br>

It seems it needed a list of ndarrays, rather than an ndarray. Of course. The perils of using undocumented things, I guess.

Christopher gravatar imageChristopher ( 2014-10-17 15:06:56 -0500 )edit

@Christopher, Ah shit yes!! I guess I missed that too. I have working code for my app and I just couldn't see what I was missing. Indeed, the descriptors need to be passed as list

bad_keypoints gravatar imagebad_keypoints ( 2014-10-18 00:21:51 -0500 )edit

3 answers

Sort by ยป oldest newest most voted
0

answered 2014-12-02 20:37:29 -0500

I have reproduced this problem... it appears to be a bug in the implementation. I will update this post if I find any solutions.

Here is a backtrace:

#0 __memmove_ssse3_back () at ../sysdeps/x86_64/multiarch/memcpy-ssse3-back.S:1549

#1 0x00007ffff55632b2 in cvflann::LshIndex<cvflann::hamminglut2>::buildIndex() () from /usr/local/lib/libopencv_flann.so.2.4

#2 0x00007ffff555e468 in void cv::flann::buildIndex_<cvflann::hamminglut2, cvflann::index<cvflann::hamminglut2=""> >(void*&, cv::Mat const&, cv::flann::IndexParams const&, cvflann::HammingLUT2 const&) () from /usr/local/lib/libopencv_flann.so.2.4

#3 0x00007ffff555f119 in cv::flann::Index::build(cv::_InputArray const&, cv::flann::IndexParams const&, cvflann::flann_distance_t) () from /usr/local/lib/libopencv_flann.so.2.4

#4 0x00007ffff4a9d964 in cv::FlannBasedMatcher::train() () from /usr/local/lib/libopencv_features2d.so.2.4

#5 0x00007ffff61cd41b in pyopencv_DescriptorMatcher_train(_object, _object, _object*) () from /usr/local/lib/python2.7/dist-packages/cv2.so

edit flag offensive delete link more
0

answered 2014-12-02 20:37:55 -0500

I have reproduced this problem... it appears to be a bug in the implementation. I will update this post if I find any solutions.

Here is a backtrace:

#0 __memmove_ssse3_back () at ../sysdeps/x86_64/multiarch/memcpy-ssse3-back.S:1549

#1 0x00007ffff55632b2 in cvflann::LshIndex<cvflann::hamminglut2>::buildIndex() () from /usr/local/lib/libopencv_flann.so.2.4

#2 0x00007ffff555e468 in void cv::flann::buildIndex_<cvflann::hamminglut2, cvflann::index<cvflann::hamminglut2=""> >(void*&, cv::Mat const&, cv::flann::IndexParams const&, cvflann::HammingLUT2 const&) () from /usr/local/lib/libopencv_flann.so.2.4

#3 0x00007ffff555f119 in cv::flann::Index::build(cv::_InputArray const&, cv::flann::IndexParams const&, cvflann::flann_distance_t) () from /usr/local/lib/libopencv_flann.so.2.4

#4 0x00007ffff4a9d964 in cv::FlannBasedMatcher::train() () from /usr/local/lib/libopencv_features2d.so.2.4

#5 0x00007ffff61cd41b in pyopencv_DescriptorMatcher_train(_object, _object, _object*) () from /usr/local/lib/python2.7/dist-packages/cv2.so

edit flag offensive delete link more
0

answered 2015-09-04 06:48:38 -0500

Hi, Christopher. Are you solved this question?
(it was a half year ago...)

I got same problems. but i got it clear.

I think, we needed take the descriptions at the type "float32".
so, it will be nice in this code bellow.

flann.add(np.float32(des2))

also we should teke same type in matching.

matches = flann.knnMatch(np.float32(des1),k=2)

see this question and the answer.
http://stackoverflow.com/questions/19...

try it. yuzu

# i don't write/speak english well.. so i hope you understand this post. ;)
edit flag offensive delete link more
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2014-10-16 22:17:43 -0500

Seen: 5,591 times

Last updated: Dec 02 '14