MLP XOR classification problem with predict method

asked 2017-02-02 08:30:33 -0500

Ihab Taher gravatar image

updated 2017-02-02 16:31:36 -0500

Note : I misused the noun of my object as I named it Digits_Classifier putting in mind I want to classify digits but I started with this simple example to try to understand OpenCV but I faced this problem here :

Does anyone have any idea of why this code produces incorrect classification labels

public class Classifier {

// This is an MLP classifier
// We will use an MLP with structure 2-2-1 to do the task of classifying XOR problem

// MLP Attributes
ANN_MLP Digits_Classifier;
Mat Layers; // A mat contains number of neurons in each layer starting from the input till the output
Mat Training_Samples; // Training data
Mat Training_Responses; // Training data labels
Mat Test_Samples; // Test data 
Mat Test_Responses; // Test data labels that will be produced by MLP
int Row_Index , Col_Index; 
float Label;

// class constructor just declares / defines attributes
public Classifier()
{
    Digits_Classifier = ANN_MLP.create();

    Layers = new Mat(3,1,CvType.CV_32FC1);
    Training_Samples = new Mat(4,2,CvType.CV_32FC1);
    Training_Responses = new Mat(4,1,CvType.CV_32FC1);
    Test_Samples = new Mat(1,2,CvType.CV_32FC1);
    Test_Responses = new Mat(1,1,CvType.CV_32FC1);
}

public void Start()
{
    // Fill in Layers Mat
    int Layers_Arr[] = new int[]{2,2,1};
    for(int i=0 ; i<3 ; i++)
        Layers.put(i,0,Layers_Arr[i]);

    Row_Index = 0;
    Col_Index = 0;

    // Fill in Training_Samples Mat
    float[] Training_Samples_Arr = new float[]{0,0,0,1,1,0,1,1};
    for(int i=0 ; i<8 ; i++) {
        Training_Samples.put(Row_Index,Col_Index,Training_Samples_Arr[i]);
        Col_Index++;
        if(Col_Index >= 2)
        {
            Row_Index++;
            Col_Index = 0;
        }
    }

    // Fill in Training_Responses Mat
    float[] Training_Responses_Arr = new float[]{0,1,1,0};
    for(int i=0 ; i<4 ; i++)
        Training_Responses.put(i,0,Training_Responses_Arr[i]);

    // Fill in Test_Samples Mat
    double[] Test_Samples_Arr = new double[] {1.0,0.0};
    for(int i=0 ; i<2 ; i++)
        Test_Samples.put(0,i,Test_Samples_Arr[i]);

    Digits_Classifier.setLayerSizes(Layers);

    // The other 2 parameters other than type are Alpha and Beta
    Digits_Classifier.setActivationFunction(ANN_MLP.SIGMOID_SYM, 1.0, 1.0);

    // Term_Criteria object is created with 3 parameters : type , Iterations_Num , Error_Limit
    TermCriteria Digits_Classifier_Term_Criteria = new TermCriteria(TermCriteria.EPS,500,0.001);
    Digits_Classifier.setTermCriteria(Digits_Classifier_Term_Criteria);

    // the other 2 parameters other than type are momentum and weight scales
    Digits_Classifier.setTrainMethod(ANN_MLP.BACKPROP,0.1,0.1);

    Digits_Classifier.train(Training_Samples,Ml.ROW_SAMPLE,Training_Responses);

    // I think this method is the source of problem I face. 
    Label = Digits_Classifier.predict(Test_Samples,Test_Responses,StatModel.RAW_OUTPUT);
}

}

  • So to summarize this is a 2-2-1 Neural net I use to perform XOR problem. I trained the network by using train method just once I then tried to predict what will the output be for test_sample such as (1 , 0) but the predict method returns 0 although it must be one and another thing is that I provided this example as a training sample what is the problem ?

  • Another thing I want to ask about is that I defined Layers to be 3x1 Mat i.e. to have 3 layers one of them is the hidden one but when I use function getweights(int Layer_Index) it accepts until Layer_Index = 4 i.e. I have 5 layers so what ?

Note : I have ... (more)

edit retag flag offensive close merge delete

Comments

1

btw: StatModel.UPDATE_MODEL should not go into the predict() method, but (maybe) into train().

again, you only need that flag, if you want to use a previously trained model with additional data.

also, we probably need to see, how you setup your network, and your data.

berak gravatar imageberak ( 2017-02-02 08:40:32 -0500 )edit
1

Thank you. I will edit this post to contain a more clear description of this network.

Ihab Taher gravatar imageIhab Taher ( 2017-02-02 09:34:07 -0500 )edit

not an answer yet, but:

  • RAW_OUTPUT is specific to SVM classifier, and will be ignored.
  • admittedly, the ANN should return correct predictions for this simple problem with default params, but doesn't (that's baaad :\ ) still, try: Digits_Classifier.setTrainMethod(ANN_MLP.BACKPROP,0.5,0.5);
  • for a multi - class problem, your train responses must be "one-hot-encoded", and you will need an output neuron per class
berak gravatar imageberak ( 2017-02-04 03:42:12 -0500 )edit

Thank you. I think it is finally working I have provided these examples and here are the responses I got ( putting in mind labeling classes 1,-1)

0.1,0.1 -> -0.95
0.2,0.2 -> -.066
0.8,0.8 -> -0.66
0.9,0.9 -> -.095
0.2,0.9 -> 0.998
0.5,0.5 -> 0.998

But I have 2 questions 1. How is Layers Mat organized as I chose 3 layers and I found Layers Mat has 5 elements i.e. 5 layers ? 2. What is weightScale parameter taken by setTrainMethod is this the learning rate ?

Ihab Taher gravatar imageIhab Taher ( 2017-02-04 09:42:21 -0500 )edit