Low accurracy from this CvANN_MLP in ByteFish Machine Learning Guide
I got the code from this link under the top answer. I whittled it down to what is below. In the pdf the code's author wrote he was getting over 90% accuracy. You can see it in this pdf on page 11. I was wondering If someone can tell me how to improve the accuracy of this program to get the over 90% accuracy the author was getting.
#include <iostream>
#include <math.h>
#include <string>
#include "cv.h"
#include "ml.h"
#include "highgui.h"
#include "opencv2/imgproc.hpp"
#include "opencv2/highgui.hpp"
#include "opencv2/core.hpp"
#include <iostream>
#include <stdlib.h>
using namespace std;
using namespace cv;
bool plotSupportVectors=true;
int numTrainingPoints=200;
int numTestPoints=2000;
int size=200;
int eq=0;
// accuracy
float evaluate(cv::Mat& predicted, cv::Mat& actual) {
assert(predicted.rows == actual.rows);
int t = 0;
int f = 0;
for(int i = 0; i < actual.rows; i++) {
float p = predicted.at<float>(i,0);
float a = actual.at<float>(i,0);
if((p >= 0.0 && a >= 0.0) || (p <= 0.0 && a <= 0.0)) {
t++;
} else {
f++;
}
}
return (t * 1.0) / (t + f);
}
// plot data and class
void plot_binary(cv::Mat& data, cv::Mat& classes, string name) {
cv::Mat plot(size, size, CV_8UC3);
plot.setTo(cv::Scalar(255.0,255.0,255.0));
for(int i = 0; i < data.rows; i++) {
float x = data.at<float>(i,0) * size;
float y = data.at<float>(i,1) * size;
if(classes.at<float>(i, 0) > 0) {
cv::circle(plot, Point(x,y), 2, CV_RGB(255,0,0),1);
} else {
cv::circle(plot, Point(x,y), 2, CV_RGB(0,255,0),1);
}
}
imshow(name, plot);
}
// function to learn
int f(float x, float y, int equation) {
switch(equation) {
case 0:
return y > sin(x*10) ? -1 : 1;
break;
case 1:
return y > cos(x * 10) ? -1 : 1;
break;
case 2:
return y > 2*x ? -1 : 1;
break;
case 3:
return y > tan(x*10) ? -1 : 1;
break;
default:
return y > cos(x*10) ? -1 : 1;
}
}
// label data with equation
cv::Mat labelData(cv::Mat points, int equation) {
cv::Mat labels(points.rows, 1, CV_32FC1);
for(int i = 0; i < points.rows; i++) {
float x = points.at<float>(i,0);
float y = points.at<float>(i,1);
labels.at<float>(i, 0) = f(x, y, equation);
}
return labels;
}
void mlp(cv::Mat& trainingData, cv::Mat& trainingClasses, cv::Mat& testData, cv::Mat& testClasses) {
cv::Mat layers = cv::Mat(4, 1, CV_32SC1);
layers.row(0) = cv::Scalar(2);
layers.row(1) = cv::Scalar(10);
layers.row(2) = cv::Scalar(15);
layers.row(3) = cv::Scalar(1);
CvANN_MLP mlp;
CvANN_MLP_TrainParams params;
CvTermCriteria criteria;
criteria.max_iter = 100;
criteria.epsilon = 0.00001f;
criteria.type = CV_TERMCRIT_ITER | CV_TERMCRIT_EPS;
params.train_method = CvANN_MLP_TrainParams::BACKPROP;
params.bp_dw_scale = 0.05f;
params.bp_moment_scale = 0.05f;
params.term_crit = criteria;
mlp.create(layers);
// train
mlp.train(trainingData, trainingClasses, cv::Mat(), cv::Mat(), params);
cv::Mat response(1, 1, CV_32FC1);
cv::Mat predicted(testClasses.rows, 1, CV_32F);
for(int i = 0; i < testData.rows; i ...
the >90% values are for the most simple equation 2 (y=2*x), a plain line. (and even there knn/dtree wins)
@berak I've seen Neural Nets used in Face recognition successfully and this seems like a lesser issue than Face Reg. Is there any way to improve the NN's on this program or is it a limit of NN's in general that they can't do this with over 90% accuracy or is it a Limit of OpenCV NN's
if you want my 2 cents, the mlp there is quite restricted. as you can see in the demo above, other ml algos fare much better. and for face reco, an svm will be much faster and better than that mlp, and a simple nn search over lbp-like features will beat that again
@berak when you say " the mlp there is quite restricted." you mean the NN classes in OpenCV or in the snippet I posted
i mean the CvANN_MLP class implemented in opencv (the one that is used in the code above) .
if you're curious about NN's and how to build different and better ones: https://class.coursera.org/neuralnets-2012-001/class/index
@berak Thanks for the info, still trying to learn linear algebra so i can learn the coursera course. You know, all those equations.
+1 for trying even ;)