# how do you save a SparseMat as binary, including the path?

The only way that I have found to write a SparseMat is with : void write( FileStorage& fs, const String& name, const SparseMat& value ) from persistence.cpp line 5493. This writes it as a YAML document and does not accept a path but only the file name. The desired SparseMat is two-dimensional with Vec5f data.

I think I can use OpenEXR organized with 5 channels as it supports arbitrary channels, but it is looking for a dense matrix and I want to reduce formatting and transmission time for what would otherwise be a very large matrix. This would probably involve a sparse to dense on the one end and a dense to sparse on the other.

Is there an example of writing a binary SparseMat out there?

Thanks Ralph

edit retag close merge delete

just out of curiosity: what's inside your sparse mat ? (and how large is it?)

( 2015-12-23 02:22:30 -0500 )edit

Sort by » oldest newest most voted

well you could try to use the FileStorage wo write xml/yml, but unfortunately it won't be able to read it back in again (reading VecfX seems limited to 4 elems).

messing with exr seems quite daring, too, so let's just roll our own storage

(note, that this is a very special case, limited to 2d / Vec5f) :

#include "opencv2/opencv.hpp"
#include <iostream>
using namespace cv;
using namespace std;

typedef Vec<float,5> Vec5f;

#include <fstream>
#define SPARSEBINARY

bool write2d(const char *fn, SparseMat &m)
{
ofstream s(fn, ios::out | ios::binary);

int h = m.hdr->size[0];
int w = m.hdr->size[1];
#ifdef SPARSEBINARY
s.write((char*)&h, sizeof(int));
s.write((char*)&w, sizeof(int));
#else // text
s << h << " " << w << endl;
#endif

for (int i=0; i<h; i++)
{
for (int j=0; j<w; j++)
{
if (0 == m.ptr(i,j,false))
continue;
Vec5f e = m.ref<Vec5f>(i,j);
#ifdef SPARSEBINARY
s.write((char*)&i, sizeof(int));
s.write((char*)&j, sizeof(int));
s.write((char*)&e, sizeof(Vec5f));
#else // text
s << i << " " << j << " ";
s << e[0] << " " << e[1] << " " << e[2] << " " << e[3] << " " << e[4] << endl;
#endif
}
}
s.close();
return true;
}

bool read2d(const char *fn, SparseMat &m)
{
ifstream s(fn, ios::out | ios::binary);
if (!s) return false;

int h=0,w=0;
#ifdef SPARSEBINARY
#else
s >> h >> w;
#endif

int sz[] = {h,w};
m = SparseMat(2,sz,CV_32FC(5));
while(!s.eof())
{
int i,j;
Vec5f e;
#ifdef SPARSEBINARY
#else
s >> i >> j >> e[0] >> e[1] >> e[2] >> e[3] >> e[4];
#endif
if (e == Vec5f::zeros()) // weird..
break;
m.ref<Vec5f>(i,j) = e;
}
s.close();
return true;
}

int main(int argc, char **argv)
{
int sz[] = {10,10};
SparseMat m(2,sz,CV_32FC(5));

m.ref<Vec5f>(2,2) = Vec5f(1,2,3,4,5);
m.ref<Vec5f>(4,7) = Vec5f(2.2,1.1,3,3,3);
m.ref<Vec5f>(7,3) = Vec5f(6,7,8.1,9,0);
write2d("my.xxx",m);

SparseMat m2;
cerr << m2.ref<Vec5f>(2,2) << endl;
cerr << m2.ref<Vec5f>(4,7) << endl;
cerr << m2.ref<Vec5f>(7,3) << endl;
cerr << m2.ref<Vec5f>(9,9) << endl; // does not exist
return 0;
}

more

Thanks. It looks like it will work.

I looked at how OpenCV handled imread and imwrite. OpenCV 3.0 documentation says that it supports reading and writing EXR. However, the encoder in grfmt_exr.hpp is too simple to handle anything but RGB and I would have to write my one encoder, at least. I had come to the conclusion that rolling my own was the only answer. Since I am more of a hacker than a software engineering, I would have had to ask someone else to write it. I will give your take a try.

The attraction that I saw in using OpenEXR was using half float. Most image processing numerical analysis issues are with the dynamic range. 10 bits of mantissa are sufficient (actually 11 with the implied first bit from the exponent) with most image processing with most sensors.

( 2015-12-23 10:26:34 -0500 )edit

Just in case some one else has that problem, I passed this on. When I copied the code, CV_32FC(5) wasn't liked by the analyzer. I typed it in again and it worked. It appears that there was a hidden special character or a Unicode issue (some other character substituted for what appeared on the screen).

( 2015-12-23 10:44:39 -0500 )edit

ouch, let me check, if VS made some utf mess of it again .. (no, wrong guess.)

( 2015-12-23 10:52:06 -0500 )edit

The write2d function seemed to work and produced a file of the size that I expected, which fits my bandwidth. Now, I have to fix the next module to use read2d. I use strings for file names. So, I added a conversion at the front of each function: int read2d(filename FileName, SparseMat &m) { const char *fn = FileName.c_str();

( 2015-12-23 11:12:15 -0500 )edit

Berak It could be a Windows 10, cut and past asynchronous innovation peculiar to my desktop. I am running the IDE in Ubuntu 12.04 in a VirtualBox. I use Code Composer Studio because of the TI targets in the system.

( 2015-12-23 11:15:03 -0500 )edit

hehe, usually , type main.cpp > main2.cpp (on a win console) does the trick of removing unicode/utf bs for me

( 2015-12-23 11:37:13 -0500 )edit

read2d appears to be working. For some reason, it doesn't like the read and write in the functions in the other project. So, I added a read2d line after my write2d line to read the file I just wrote. I put m.zncount() and got my expected 12164. This is out of 197192 pixels in the dense Mat. According to my calculations, I should get between 12007 and 12167. (Which is why I am using SparseMat.) The #includes all seem OK. The .cpp was copied and pasted from the project where it works. I assume it is some kind of linkage problem, but I don't see it. I assume the residual problem has nothing to do with OpenCV and my question has been answered. Thanks a lot.

( 2015-12-23 14:40:42 -0500 )edit

Official site

GitHub

Wiki

Documentation