Ask Your Question
1

PCA + SVM ?

asked 2016-06-15 12:37:24 -0600

RPH gravatar image

Hi, I am trying to reduce the dimensionality of my feature set using PCA and then classification using SVM.

I have created and populated my feature matrix and label matrix :

  Mat labels(875, 1, CV_32F);

  Mat trainingData(875, 23040, CV_32F);

Where each row corresponds to an image and each column is a feature. The features are histogram bins (256) from multiple regions in the image (90 regions). 90x256 = 23040.

I now want to reduce the 23040 features to something less before training, say 512 features. I have this:

/*Reduce dimensionality using PCA*/

Mat projection_result;

PCA pca(trainingData, Mat(), CV_PCA_DATA_AS_ROW,512);

pca.project(trainingData, projection_result);

/*Classification*/

CvSVMParams params;
params.svm_type = CvSVM::C_SVC;
params.kernel_type = CvSVM::RBF;

CvSVM SVM;
SVM.train_auto(projection_result, labels, Mat(), Mat(), params);

SVM.save("trainedData_test.xml");

My question is, is this correct ? And how would I now use prediction with PCA ?

/*Load SVM*/

CvSVM SVM;
SVM.load("trainedData_test.xml");


/*Populate test matrix*/

  Mat testData(1, 23040, CV_32F);

/*PCA ???*/

/*Classify*/

int label = SVM.predict(projection_result ???);

Thanks !

edit retag flag offensive close merge delete

1 answer

Sort by ยป oldest newest most voted
5

answered 2016-06-16 04:16:28 -0600

berak gravatar image

updated 2016-06-17 01:35:33 -0600

"is this correct ?"

yes, looks fine.

"And how would I now use prediction with PCA ? "

the very same way you treated your train data. you will have to keep the pca object around (at least the transposed eigenvectors and the mean) and project your test vector into pca space:

Mat testData(1, 23040, CV_32F);
Mat test_projected;
pca.project(testData, test_projected); // should result in a 512 x 1 Mat
int label = SVM.predict(test_projected);
// btw, naming your instance "SVM" is a bad idea, since there is already a class with that name

(actually, all PCA::project() does is: result = (input - mean) * eigenvecs.t())

[Edit:]

since you probably want to save the trained svm model, you need to save the PCA, too:

// to disk:
FileStorage fs("pca.xml", FileStorage::WRITE);
pca.write(fs);
fs.release(); // flush

// and back:
PCA pca;
FileStorage fs("pca.xml", FileStorage::READ);
pca.read(fs.root());
fs.release();
edit flag offensive delete link more

Comments

Thanks for the answer !

How would I go about using it if I wanted to save the SVM for later testing ? I assume one cannot save the whole PCA object (like with the SVM), so is it easy to save and re-use the mean and eigenvectors ?

Thanks !

RPH gravatar imageRPH ( 2016-06-16 16:26:23 -0600 )edit

^^ ah, right, i knew there was something amiss !

(please see edit)

berak gravatar imageberak ( 2016-06-17 00:28:09 -0600 )edit

Thanks for all the help guys ! I'll test it out in the coming days and post back if it gives any issues...Thanks

RPH gravatar imageRPH ( 2016-06-18 13:07:22 -0600 )edit

yes,please report back, curious, how you're doing ;)

berak gravatar imageberak ( 2016-06-18 13:22:38 -0600 )edit

Hi, I have tried to implement the code however I get the error:

"cv::PCA has no member write"

What is the problem do you think ? Thanks

RPH gravatar imageRPH ( 2016-06-28 13:01:50 -0600 )edit

@RPH what is your OpenCV version?

sturkmen gravatar imagesturkmen ( 2016-06-28 13:05:29 -0600 )edit

Hi, it is version 2.4.12

RPH gravatar imageRPH ( 2016-06-28 13:09:26 -0600 )edit

the code in the answer should work! please share the code you tried

sturkmen gravatar imagesturkmen ( 2016-06-28 13:20:51 -0600 )edit

@sturkmen I am using 32bit version if that makes a difference. The problem is with the .write line.

RPH gravatar imageRPH ( 2016-06-28 13:23:59 -0600 )edit

just tried like:

PCA pca;
FileStorage fs("pca.xml", FileStorage::WRITE);
pca.write(fs);
fs.release(); // flush
sturkmen gravatar imagesturkmen ( 2016-06-28 13:33:16 -0600 )edit

Question Tools

1 follower

Stats

Asked: 2016-06-15 12:37:24 -0600

Seen: 1,683 times

Last updated: Jun 17 '16