Ask Your Question
2

linear least-squares for large-scale dense matrix

asked 2013-08-08 04:48:58 -0500

grtwa gravatar image

updated 2013-08-08 20:50:42 -0500

I use the function "cv::solve(src1, src2, dst, DECOMP_SVD)" to solve a least-squares problem (i.e., src1 * dst = src2). The size of src1 is 29030 * 7809 and the size of src2 is 29030 * 122. Both types of src1 and src2 are CV_32FC1. OpenCV crashes at the 1370 line of "lapack.cpp" and the code of this line is "buffer.allocate(bufsize)". "buffer" is a "AutoBuffer< uchar >" and "bufsize" is 1151016404. Is this problem caused by out-of-memory? My physical memory is 8G and operating system is 64bit Win7.

Mat A(29030,7809,CV_32FC1);
Mat b(29030,122,CV_32FC1);
for(i=0;i<29030;i++)
{
    for(j=0;j<7809;j++)
        A.at<float>(i,j)=float(i*7809+j);

    for(j=0;j<122;j++)
        b.at<float>(i,j)=float(i*122+j);
}

Mat x;
solve(A,b,x,DECOMP_SVD);
edit retag flag offensive close merge delete

Comments

1

My c++ knowledge is not the best, you know. But I think your Matrices are getting allocated on the stack which is kind of small, but fast. You should maybe create them on the heap. So try it like this: Mat *A = new Mat(29030, 7809, CV_32FC1); Mat *B = new Mat(29030, 122, CV_32FC1);

I could of course be totally wrong, but at the moment you are trying to allocate 878 MByte on the stack :P

Moster gravatar imageMoster ( 2013-08-09 01:17:21 -0500 )edit
2

Thank you. I have solved this problem by convert the compiling platform to x64. On x86 platform, only 2G physical memory can be used. It's my fault, not OpenCV :)

grtwa gravatar imagegrtwa ( 2013-08-09 01:28:51 -0500 )edit
1

Did you try my solution? Im curious if that actually works.

Moster gravatar imageMoster ( 2013-08-09 02:21:04 -0500 )edit
1

If this problem is solved, please use the answer functionality, add your own solution and accept it. This way the problem will appeared solved and people could reuse your solution on a later time.

StevenPuttemans gravatar imageStevenPuttemans ( 2013-08-09 03:44:45 -0500 )edit
1

@Moster, I did not try because I think automatic variables are created on the heap. Any way, problem is gone :)

grtwa gravatar imagegrtwa ( 2013-08-09 04:31:07 -0500 )edit
1

@StevenPuttemans, "New users must wait 2 days before answering their own question." I will post an answer later. Thank you for reminding.

grtwa gravatar imagegrtwa ( 2013-08-09 04:35:41 -0500 )edit

2 answers

Sort by ยป oldest newest most voted
2

answered 2013-08-11 23:36:52 -0500

grtwa gravatar image

I have solved this problem by convert the compiling platform to x64. On x86 platform, only 2G physical memory can be used.

edit flag offensive delete link more
3

answered 2013-08-13 01:05:22 -0500

Michael Burdinov gravatar image

SVD method has very high numerical robustness, but this robustness has its price both in space and in time. If you don't expect a degenerate case you can you much lighter methods that should be able to run on x86. For example:

solve(A.t()*A, A.t()*b, x, DECOMP_CHOLESKY);
edit flag offensive delete link more

Comments

Thanks, I need deal with some degenerate case. Before using SVD, I intended to use QR. But DECOMP_QR seems to equate with DECOMP_SVD in OpenCV2.4.5.

grtwa gravatar imagegrtwa ( 2013-08-13 02:27:05 -0500 )edit
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2013-08-08 04:48:58 -0500

Seen: 4,124 times

Last updated: Aug 13 '13