Ask Your Question
0

Does SparseMat accept large indices? [closed]

asked 2018-03-27 02:42:17 -0500

I did:

FileStorage fsp("sparse.yml", FileStorage::WRITE);

fsp << "image";

fsp << sparse_mat;

and have got the file content:

%YAML:1.0

image: !!opencv-sparse-matrix

sizes: [ 10000, 729000 ]

dt: u

data: [ 145, 0, 1, 45, 1, -1, 245, 0, 1, 1, 1, 2, 1, 3, 1, 4, 1, 5, 1, 39, 1, 40, 1, 41, 1, 42, 1, 43, 1, 44, 1, 45, 1, 46, 1, 47, 1, 48, 1, 49, 1, 50, 1, 84, 1, 85, 1, 86, 1, 87, 1, 88, 1, 89, 1, 90, 1, 91, 1, 92, 1, 93, 1, 94, 1, 95, 1, 129, 1, 130, 1, 131, 1, 132, 1, 133, 1, 134, 1, 135, 1, 136, 1, 137, 1, 138, 1, 139, 1, 140, 1, -1, 246, 135, 1, 136, 1, 137, 1, 138, 1, 139, 1, 140, 1, 174, 1, 175, 1, 176, 1, 177, 1, 178, 1, 179, 1, 180, 1, 181, 1, 182, 1, ...

The matrix contains only 1 in some elements (unsigned char format). So far as I understand, the file should be filled by triplets: 2 indices + value. 145, 0, 1 looks fine. Further - complete nonsense. In any case -1 shouldn't be here. I blame large indices (see 'sizes:'). Mat simply crashes with such numbers. The program aborts at start. SparseMat works, but this way. May it be the reason and what is the limit for sizes?

edit retag flag offensive reopen merge delete

Closed for the following reason the question is answered, right answer was accepted by ya_ocv_user
close date 2018-03-27 12:23:54.340544

1 answer

Sort by ยป oldest newest most voted
1

answered 2018-03-27 04:31:15 -0500

LBerger gravatar image

updated 2018-03-27 07:23:01 -0500

I don't understand your problem If change code (answer) :

const int dims = 2;
int size[2] = { pow(2,31)-1,pow(2,31)-1 };
SparseMat sparse_mat(dims, size, CV_32F), sparse_mat2;
for (int i = 0; i < 10; i++)
{
    int idx[dims];
    for (int k = 0; k < dims; k++)
        idx[k] = size[k] - 10*k-1-i;
    sparse_mat.ref<float>(idx) += 1.f;
}
SparseMatConstIterator_<float> it = sparse_mat.begin<float>(),
    it_end = sparse_mat.end<float>();
cout << "****************WRITE \n";
int dim = sparse_mat.dims();
for (; it != it_end; ++it)
{
    const SparseMat::Node* n = it.node();
    for (int i = 0; i < dim; i++)
        cout << n->idx[i] << "\t";
    cout << " -> " << it.value<float>() << "\n";

}
FileStorage fsp("sparse.yml", FileStorage::WRITE);
fsp << "image";
fsp << sparse_mat;
fsp.release();
FileStorage fsp2("sparse.yml", FileStorage::READ);
fsp2["image"] >> sparse_mat2;
fsp2.release();
cout << "****************READ \n";
SparseMatConstIterator_<float> it2 = sparse_mat2.begin<float>(),
    it2_end = sparse_mat2.end<float>();
dim = sparse_mat2.dims();
for (; it2 != it2_end; ++it2)
{
    const SparseMat::Node* n = it2.node();
    for (int i = 0; i < dim; i++)
        cout << n->idx[i] << "\t";
    cout << " -> " << it2.value<float>() << "\n";

}

answer is

****************WRITE
2147483639      2147483629       -> 1
2147483643      2147483633       -> 1
2147483638      2147483628       -> 1
2147483642      2147483632       -> 1
2147483646      2147483636       -> 1
2147483637      2147483627       -> 1
2147483641      2147483631       -> 1
2147483645      2147483635       -> 1
2147483640      2147483630       -> 1
2147483644      2147483634       -> 1
****************READ
2147483639      2147483629       -> 1
2147483643      2147483633       -> 1
2147483642      2147483632       -> 1
2147483646      2147483636       -> 1
2147483638      2147483628       -> 1
2147483645      2147483635       -> 1
2147483637      2147483627       -> 1
2147483641      2147483631       -> 1
2147483640      2147483630       -> 1
2147483644      2147483634       -> 1

Now inside code hash value is an int64 (MSVC 2017 Win64) and index is an unsigned int

yml file is :

%YAML:1.0

image: !!opencv-sparse-matrix sizes: [ 2147483647, 2147483647 ] dt: f data: [ 2147483637, 2147483627, 1., -1, 2147483638, 2147483628, 1., -1, 2147483639, 2147483629, 1., -1, 2147483640, 2147483630, 1., -1, 2147483641, 2147483631, 1., -1, 2147483642, 2147483632, 1., -1, 2147483643, 2147483633, 1., -1, 2147483644, 2147483634, 1., -1, 2147483645, 2147483635, 1., -1, 2147483646, 2147483636, 1. ]

About -1 between data it's here Value between data is equal to -dims+1 for dim=2 it is -1 and dim=3 -2 ...

Why a separator? I don't know!

edit flag offensive delete link more

Comments

So your answer is Yes. Very well. Hence I need to search for error in my program. Thank you for correction about data format in the file. Nevertheless, in my output -1 appears irregularly.

ya_ocv_user gravatar imageya_ocv_user ( 2018-03-27 09:16:45 -0500 )edit

I have an idea why separator is needed. For data compression. Look here (I inserted Line Feed manually): data: [ 145, 0, 1, -1,

245, 0, 1, 1, 1, 2, 1, 3, 1, 4, 1, 45, 1, 46, 1, 47, 1, 48, 1, 49, 1, 90, 1, 91, 1, 92, 1, 93, 1, 94, 1, -1,

345, 0, 1, 1, 1, 2, 1, 3, 1, 4, 1, 45, 1, 46, 1, 47, 1, 48, 1, 49, 1, 90, 1, 91, 1, 92, 1, 93, 1, 94, 1, -1,

445, 0, 1, 1, 1, 2, 1, 3, 1, 4, 1, 45, 1, 46, 1, 47, 1, 48, 1, 49, 1, 90, 1, 91, 1, 92, 1, 93, 1, 94, 1, -1,

545, 0, 1, 1, 1, 2, 1, 3, 1, 4, 1, 45, 1, 46, 1, 47, 1, 48, 1, 49, 1, 90, 1, 91, 1, 92, 1, 93, 1, 94, 1 ...(more)

ya_ocv_user gravatar imageya_ocv_user ( 2018-03-27 12:20:15 -0500 )edit

can you post full yml file in your question?

LBerger gravatar imageLBerger ( 2018-03-27 13:31:06 -0500 )edit

Full file is large. 520 KB. I think it's almost certain that compression is used. Post here the beginning and the end of another matrix.

%YAML:1.0

image: !!opencv-sparse-matrix

sizes: [ 10000, 216000 ]

dt: u

data: [ 145, 0, 1, 30, 1, -1,

ya_ocv_user gravatar imageya_ocv_user ( 2018-03-28 05:33:34 -0500 )edit

245, 0, 1, 1, 1, 2, 1, 3, 1, 20, 1, 21, 1, 22, 1, 23, 1, 26, 1, 27, 1, 28, 1, 29, 1, 30, 1, 31, 1, 32, 1, 33, 1, 40, 1, 41, 1, 42, 1, 43, 1, 46, 1, 47, 1, 48, 1, 49, 1, 50, 1, 51, 1, 52, 1, 53, 1, 66, 1, 67, 1, 68, 1, 69, 1, 70, 1, 71, 1, 72, 1, 73, 1, -1,

246, 60, 1, 61, 1, 62, 1, 63, 1, 86, 1, 87, 1, 88, 1, 89, 1, 90, 1, 91, 1, 92, 1, 93, 1, -1,

247, 80, 1, 81, 1, 82, 1, 83, 1, 106, 1, 107, 1, 108, 1, 109, 1, 110, 1, 111, 1, 112, 1, 113, 1, -1,

248, 99, 1, 100, 1, 101, 1, 102, 1, 103, 1, 126, 1, 127, 1, 128, 1, 129, 1, 130, 1, 131, 1, 132 ...(more)

ya_ocv_user gravatar imageya_ocv_user ( 2018-03-28 05:35:09 -0500 )edit

9843, 114, 1, 115, 1, 116, 1, 117, 1, 118, 1, 119, 1, -1,

9844, 94, 1, 95, 1, 96, 1, 97, 1, 98, 1, 99, 1, -1,

9845, 54, 1, 55, 1, 56, 1, 57, 1, 58, 1, 59, 1, 74, 1, 75, 1, 76, 1, 77, 1, 78, 1, 79, 1, -1,

9936, 278, 1, 279, 1, -1,

9937, 258, 1, 259, 1, -1,

9938, 237, 1, 238, 1, 239, 1, -1,

9939, 196, 1, 197, 1, 198, 1, 199, 1, 217, 1, 218, 1, 219, 1, -1,

9940, 176, 1, 177, 1, 178, 1, 179, 1, -1,

9941, 156, 1, 157, 1, 158, 1, 159, 1, -1,

9942, 136, 1, 137, 1, 138, 1, 139, 1, -1,

9943, 116, 1, 117, 1, 118, 1, 119, 1, -1,

9944, 96, 1, 97, 1, 98, 1, 99, 1, -1,

9945, 56, 1, 57, 1, 58, 1, 59, 1, 76 ...(more)

ya_ocv_user gravatar imageya_ocv_user ( 2018-03-28 05:37:14 -0500 )edit

Question Tools

1 follower

Stats

Asked: 2018-03-27 02:42:17 -0500

Seen: 31 times

Last updated: Mar 27 '18