Low accurracy from this CvANN_MLP in ByteFish Machine Learning Guide

asked 2014-05-22 19:44:41 -0600

joeish80829 gravatar image

updated 2017-08-01 18:40:52 -0600

I got the code from this link under the top answer. I whittled it down to what is below. In the pdf the code's author wrote he was getting over 90% accuracy. You can see it in this pdf on page 11. I was wondering If someone can tell me how to improve the accuracy of this program to get the over 90% accuracy the author was getting.

            #include <iostream>
            #include <math.h>
            #include <string>
            #include "cv.h"
            #include "ml.h"
            #include "highgui.h"
            #include "opencv2/imgproc.hpp"
            #include "opencv2/highgui.hpp"
            #include "opencv2/core.hpp"
            #include <iostream>
            #include <stdlib.h>

            using namespace std;
            using namespace cv;

            bool plotSupportVectors=true;
            int numTrainingPoints=200;
            int numTestPoints=2000;
            int size=200;
            int eq=0;

            // accuracy
            float evaluate(cv::Mat& predicted, cv::Mat& actual) {
            assert(predicted.rows == actual.rows);
            int t = 0;
            int f = 0;
            for(int i = 0; i < actual.rows; i++) {
            float p = predicted.at<float>(i,0);
            float a = actual.at<float>(i,0);
            if((p >= 0.0 && a >= 0.0) || (p <= 0.0 && a <= 0.0)) {
            t++;
            } else {
            f++;
            }
            }
            return (t * 1.0) / (t + f);
            }

            // plot data and class
            void plot_binary(cv::Mat& data, cv::Mat& classes, string name) {
            cv::Mat plot(size, size, CV_8UC3);
            plot.setTo(cv::Scalar(255.0,255.0,255.0));
            for(int i = 0; i < data.rows; i++) {

            float x = data.at<float>(i,0) * size;
            float y = data.at<float>(i,1) * size;

            if(classes.at<float>(i, 0) > 0) {
            cv::circle(plot, Point(x,y), 2, CV_RGB(255,0,0),1);
            } else {
            cv::circle(plot, Point(x,y), 2, CV_RGB(0,255,0),1);
            }
            }
            imshow(name, plot);
            }

            // function to learn
            int f(float x, float y, int equation) {
            switch(equation) {
            case 0:
            return y > sin(x*10) ? -1 : 1;
            break;
            case 1:
            return y > cos(x * 10) ? -1 : 1;
            break;
            case 2:
            return y > 2*x ? -1 : 1;
            break;
            case 3:
            return y > tan(x*10) ? -1 : 1;
            break;
            default:
            return y > cos(x*10) ? -1 : 1;
            }
            }

            // label data with equation
            cv::Mat labelData(cv::Mat points, int equation) {
            cv::Mat labels(points.rows, 1, CV_32FC1);
            for(int i = 0; i < points.rows; i++) {
            float x = points.at<float>(i,0);
            float y = points.at<float>(i,1);
            labels.at<float>(i, 0) = f(x, y, equation);
            }
            return labels;
            }


            void mlp(cv::Mat& trainingData, cv::Mat& trainingClasses, cv::Mat& testData, cv::Mat& testClasses) {

            cv::Mat layers = cv::Mat(4, 1, CV_32SC1);

            layers.row(0) = cv::Scalar(2);
            layers.row(1) = cv::Scalar(10);
            layers.row(2) = cv::Scalar(15);
            layers.row(3) = cv::Scalar(1);

            CvANN_MLP mlp;
            CvANN_MLP_TrainParams params;
            CvTermCriteria criteria;
            criteria.max_iter = 100;
            criteria.epsilon = 0.00001f;
            criteria.type = CV_TERMCRIT_ITER | CV_TERMCRIT_EPS;
            params.train_method = CvANN_MLP_TrainParams::BACKPROP;
            params.bp_dw_scale = 0.05f;
            params.bp_moment_scale = 0.05f;
            params.term_crit = criteria;

            mlp.create(layers);

            // train
            mlp.train(trainingData, trainingClasses, cv::Mat(), cv::Mat(), params);

            cv::Mat response(1, 1, CV_32FC1);
            cv::Mat predicted(testClasses.rows, 1, CV_32F);
            for(int i = 0; i < testData.rows; i ...
(more)
edit retag flag offensive close merge delete

Comments

the >90% values are for the most simple equation 2 (y=2*x), a plain line. (and even there knn/dtree wins)

berak gravatar imageberak ( 2014-05-23 00:09:06 -0600 )edit

@berak I've seen Neural Nets used in Face recognition successfully and this seems like a lesser issue than Face Reg. Is there any way to improve the NN's on this program or is it a limit of NN's in general that they can't do this with over 90% accuracy or is it a Limit of OpenCV NN's

joeish80829 gravatar imagejoeish80829 ( 2014-05-23 11:37:16 -0600 )edit

if you want my 2 cents, the mlp there is quite restricted. as you can see in the demo above, other ml algos fare much better. and for face reco, an svm will be much faster and better than that mlp, and a simple nn search over lbp-like features will beat that again

berak gravatar imageberak ( 2014-05-23 11:58:39 -0600 )edit

@berak when you say " the mlp there is quite restricted." you mean the NN classes in OpenCV or in the snippet I posted

joeish80829 gravatar imagejoeish80829 ( 2014-05-23 14:57:42 -0600 )edit

i mean the CvANN_MLP class implemented in opencv (the one that is used in the code above) .

if you're curious about NN's and how to build different and better ones: https://class.coursera.org/neuralnets-2012-001/class/index

berak gravatar imageberak ( 2014-05-24 00:52:37 -0600 )edit

@berak Thanks for the info, still trying to learn linear algebra so i can learn the coursera course. You know, all those equations.

joeish80829 gravatar imagejoeish80829 ( 2014-05-24 23:30:23 -0600 )edit

+1 for trying even ;)

berak gravatar imageberak ( 2014-05-24 23:36:49 -0600 )edit