Ask Your Question

Revision history [back]

MLP XOR classification problem with predict method

Does anyone have any idea of why this code produces incorrect classification labels

ANN_MLP Digits_Classifier;

Digits_Classifier = ANN_MLP.create();

Digits_Classifier.setLayerSizes(Layers); // Layers is a 3x1 matrix.

Digits_Classifier.setActivationFunction(ANN_MLP.SIGMOID_SYM, 1.0, 1.0); // the other two parameters other than the type are Alpha and Beta consts.

TermCriteria Digits_Classifier_Term_Criteria = new TermCriteria(TermCriteria.EPS,500,0.001);

Digits_Classifier.setTermCriteria(Digits_Classifier_Term_Criteria);

Digits_Classifier.setTrainMethod(ANN_MLP.BACKPROP,0.1,0.1); // the 2 other parameters other than type are momentum and weight scales.

Digits_Classifier.train(Training_Samples,Ml.ROW_SAMPLE,Training_Responses); // this is the function that trains MLP the first parameter is a 4x2 mat and the third one is a 4x2 mat and both of them are defined somewhere

Label = Digits_Classifier.predict(Test_Samples,Test_Responses,StatModel.UPDATE_MODEL); // this predict function is to test MLP on inputs not seen before // Test_sanples is an 1x2 mat and Test_Responses is an 1x1 mat.

My problem is Label ( predict returned value ) is always 0 even if I test with 1,0 which included in training with output 1.

MLP XOR classification problem with predict method

Does anyone have any idea of why this code produces incorrect classification labels

ANN_MLP Digits_Classifier;

Digits_Classifier = ANN_MLP.create();

Digits_Classifier.setLayerSizes(Layers); // Layers is a 3x1 matrix.

Digits_Classifier.setActivationFunction(ANN_MLP.SIGMOID_SYM, 1.0, 1.0); // the other two parameters other than the type are Alpha and Beta consts.

TermCriteria Digits_Classifier_Term_Criteria = new TermCriteria(TermCriteria.EPS,500,0.001);

Digits_Classifier.setTermCriteria(Digits_Classifier_Term_Criteria);

Digits_Classifier.setTrainMethod(ANN_MLP.BACKPROP,0.1,0.1); // the 2 other parameters other than type are momentum and weight scales.

Digits_Classifier.train(Training_Samples,Ml.ROW_SAMPLE,Training_Responses); // this is the function that trains MLP the first parameter is a 4x2 mat and the third one is a 4x2 4x1 mat and both of them are defined somewhere

Label = Digits_Classifier.predict(Test_Samples,Test_Responses,StatModel.UPDATE_MODEL); // this predict function is to test MLP on inputs not seen before // Test_sanples is an 1x2 mat and Test_Responses is an 1x1 mat.mat. // Label is a float variable that as I understand (but I am not sure) is the class label of this test sample but here is another problem how it will act if I have many test samples.

My problem is Label ( predict returned value ) is always 0 even if I test with 1,0 which included in training with output 1.

click to hide/show revision 3
No.3 Revision

updated 2017-02-02 08:38:15 -0600

berak gravatar image

MLP XOR classification problem with predict method

Does anyone have any idea of why this code produces incorrect classification labels

ANN_MLP Digits_Classifier;

Digits_Classifier; Digits_Classifier = ANN_MLP.create();

ANN_MLP.create(); Digits_Classifier.setLayerSizes(Layers); // Layers is a 3x1 matrix.

matrix. Digits_Classifier.setActivationFunction(ANN_MLP.SIGMOID_SYM, 1.0, 1.0); // the other two parameters other than the type are Alpha and Beta consts.

consts. TermCriteria Digits_Classifier_Term_Criteria = new TermCriteria(TermCriteria.EPS,500,0.001);

Digits_Classifier.setTermCriteria(Digits_Classifier_Term_Criteria);

TermCriteria(TermCriteria.EPS,500,0.001); Digits_Classifier.setTermCriteria(Digits_Classifier_Term_Criteria); Digits_Classifier.setTrainMethod(ANN_MLP.BACKPROP,0.1,0.1); // the 2 other parameters other than type are momentum and weight scales.

scales. Digits_Classifier.train(Training_Samples,Ml.ROW_SAMPLE,Training_Responses); // this is the function that trains trains MLP the first parameter is a 4x2 mat mat and the third one is a 4x1 mat and both of them are defined somewhere

somewhere Label = Digits_Classifier.predict(Test_Samples,Test_Responses,StatModel.UPDATE_MODEL); // this predict function is to test MLP on inputs not seen before // Test_sanples is an 1x2 mat and Test_Responses is an 1x1 mat. // Label is a float variable that as I understand (but I am not sure) is the class label of this test sample sample

but here is another problem how it will act if I have many test samples.

My problem is Label ( predict returned value ) is always 0 even if I test with 1,0 which included in training with output 1.

MLP XOR classification problem with predict method

Does anyone have any idea of why this code produces incorrect classification labels

public class Classifier {

// This is an MLP classifier
// We will use an MLP with structure 2-2-1 to do the task of classifying XOR problem

// MLP Attributes
ANN_MLP Digits_Classifier;
 Mat Layers; // A mat contains number of neurons in each layer starting from the input till the output
Mat Training_Samples; // Training data
Mat Training_Responses; // Training data labels
Mat Test_Samples; // Test data 
Mat Test_Responses; // Test data labels that will be produced by MLP
int Row_Index , Col_Index; 
float Label;

// class constructor just declares / defines attributes
public Classifier()
{
    Digits_Classifier = ANN_MLP.create();

Digits_Classifier.setLayerSizes(Layers); //  Layers is a 3x1 matrix.

Digits_Classifier.setActivationFunction(ANN_MLP.SIGMOID_SYM, 1.0, 1.0); // the = new Mat(3,1,CvType.CV_32FC1);
    Training_Samples = new Mat(4,2,CvType.CV_32FC1);
    Training_Responses = new Mat(4,1,CvType.CV_32FC1);
    Test_Samples = new Mat(1,2,CvType.CV_32FC1);
    Test_Responses = new Mat(1,1,CvType.CV_32FC1);
}

public void Start()
{
    // Fill in Layers Mat
    int Layers_Arr[] = new int[]{2,2,1};
    for(int i=0 ; i<3 ; i++)
        Layers.put(i,0,Layers_Arr[i]);

    Row_Index = 0;
    Col_Index = 0;

    // Fill in Training_Samples Mat
    float[] Training_Samples_Arr = new float[]{0,0,0,1,1,0,1,1};
    for(int i=0 ; i<8 ; i++) {
        Training_Samples.put(Row_Index,Col_Index,Training_Samples_Arr[i]);
        Col_Index++;
        if(Col_Index >= 2)
        {
            Row_Index++;
            Col_Index = 0;
        }
    }

    // Fill in Training_Responses Mat
    float[] Training_Responses_Arr = new float[]{0,1,1,0};
    for(int i=0 ; i<4 ; i++)
        Training_Responses.put(i,0,Training_Responses_Arr[i]);

    // Fill in Test_Samples Mat
    double[] Test_Samples_Arr = new double[] {1.0,0.0};
    for(int i=0 ; i<2 ; i++)
        Test_Samples.put(0,i,Test_Samples_Arr[i]);

    Digits_Classifier.setLayerSizes(Layers);

    // The other two 2 parameters other than  the type are Alpha and Beta consts.

Beta
    Digits_Classifier.setActivationFunction(ANN_MLP.SIGMOID_SYM, 1.0, 1.0);

    // Term_Criteria object is created with 3 parameters : type , Iterations_Num , Error_Limit
    TermCriteria Digits_Classifier_Term_Criteria = new TermCriteria(TermCriteria.EPS,500,0.001);
  Digits_Classifier.setTermCriteria(Digits_Classifier_Term_Criteria);

Digits_Classifier.setTrainMethod(ANN_MLP.BACKPROP,0.1,0.1); // the     // the other 2 other parameters other than type are momentum and weight scales.

Digits_Classifier.train(Training_Samples,Ml.ROW_SAMPLE,Training_Responses); // scales
    Digits_Classifier.setTrainMethod(ANN_MLP.BACKPROP,0.1,0.1);

    Digits_Classifier.train(Training_Samples,Ml.ROW_SAMPLE,Training_Responses);

    // I think this method is the source of problem I face. 
    Label = Digits_Classifier.predict(Test_Samples,Test_Responses,StatModel.RAW_OUTPUT);
}

}

so to summarize this is the function a 2-2-1 Neural net I use to perform XOR problem. I trained the network by using train method just once I then tried to predict what will the output be for test_sample such as 1 , 0 but the predict method returns 0 although it must be one and another thing is that trains MLP the first parameter I provided this example as a training sample what is a 4x2 mat and the third one is a 4x1 mat and both of them are defined somewhere Label = Digits_Classifier.predict(Test_Samples,Test_Responses,StatModel.UPDATE_MODEL); // this predict function is to test MLP on inputs not seen before // Test_sanples is an 1x2 mat and Test_Responses is an 1x1 mat. // Label is a float variable that as I understand (but I am not sure) is the class label of this test sample

but here is another the problem how it will act if I have many test samples.?

My problem is Label ( predict returned value ) is always 0 even if I test with 1,0 which included in training with output 1.

MLP XOR classification problem with predict method

Note : I misused the noun of my object as I named it Digits_Classifier putting in mind I want to classify digits but I started with this simple example to try to understand OpenCV but I faced this problem here :

Does anyone have any idea of why this code produces incorrect classification labels

public class Classifier {

// This is an MLP classifier
// We will use an MLP with structure 2-2-1 to do the task of classifying XOR problem

// MLP Attributes
ANN_MLP Digits_Classifier;
Mat Layers; // A mat contains number of neurons in each layer starting from the input till the output
Mat Training_Samples; // Training data
Mat Training_Responses; // Training data labels
Mat Test_Samples; // Test data 
Mat Test_Responses; // Test data labels that will be produced by MLP
int Row_Index , Col_Index; 
float Label;

// class constructor just declares / defines attributes
public Classifier()
{
    Digits_Classifier = ANN_MLP.create();

    Layers = new Mat(3,1,CvType.CV_32FC1);
    Training_Samples = new Mat(4,2,CvType.CV_32FC1);
    Training_Responses = new Mat(4,1,CvType.CV_32FC1);
    Test_Samples = new Mat(1,2,CvType.CV_32FC1);
    Test_Responses = new Mat(1,1,CvType.CV_32FC1);
}

public void Start()
{
    // Fill in Layers Mat
    int Layers_Arr[] = new int[]{2,2,1};
    for(int i=0 ; i<3 ; i++)
        Layers.put(i,0,Layers_Arr[i]);

    Row_Index = 0;
    Col_Index = 0;

    // Fill in Training_Samples Mat
    float[] Training_Samples_Arr = new float[]{0,0,0,1,1,0,1,1};
    for(int i=0 ; i<8 ; i++) {
        Training_Samples.put(Row_Index,Col_Index,Training_Samples_Arr[i]);
        Col_Index++;
        if(Col_Index >= 2)
        {
            Row_Index++;
            Col_Index = 0;
        }
    }

    // Fill in Training_Responses Mat
    float[] Training_Responses_Arr = new float[]{0,1,1,0};
    for(int i=0 ; i<4 ; i++)
        Training_Responses.put(i,0,Training_Responses_Arr[i]);

    // Fill in Test_Samples Mat
    double[] Test_Samples_Arr = new double[] {1.0,0.0};
    for(int i=0 ; i<2 ; i++)
        Test_Samples.put(0,i,Test_Samples_Arr[i]);

    Digits_Classifier.setLayerSizes(Layers);

    // The other 2 parameters other than type are Alpha and Beta
    Digits_Classifier.setActivationFunction(ANN_MLP.SIGMOID_SYM, 1.0, 1.0);

    // Term_Criteria object is created with 3 parameters : type , Iterations_Num , Error_Limit
    TermCriteria Digits_Classifier_Term_Criteria = new TermCriteria(TermCriteria.EPS,500,0.001);
    Digits_Classifier.setTermCriteria(Digits_Classifier_Term_Criteria);

    // the other 2 parameters other than type are momentum and weight scales
    Digits_Classifier.setTrainMethod(ANN_MLP.BACKPROP,0.1,0.1);

    Digits_Classifier.train(Training_Samples,Ml.ROW_SAMPLE,Training_Responses);

    // I think this method is the source of problem I face. 
    Label = Digits_Classifier.predict(Test_Samples,Test_Responses,StatModel.RAW_OUTPUT);
}

}

so to summarize this is a 2-2-1 Neural net I use to perform XOR problem. I trained the network by using train method just once I then tried to predict what will the output be for test_sample such as 1 , 0 but the predict method returns 0 although it must be one and another thing is that I provided this example as a training sample what is the problem ?

MLP XOR classification problem with predict method

Note : I misused the noun of my object as I named it Digits_Classifier putting in mind I want to classify digits but I started with this simple example to try to understand OpenCV but I faced this problem here :

Does anyone have any idea of why this code produces incorrect classification labels

public class Classifier {

// This is an MLP classifier
// We will use an MLP with structure 2-2-1 to do the task of classifying XOR problem

// MLP Attributes
ANN_MLP Digits_Classifier;
Mat Layers; // A mat contains number of neurons in each layer starting from the input till the output
Mat Training_Samples; // Training data
Mat Training_Responses; // Training data labels
Mat Test_Samples; // Test data 
Mat Test_Responses; // Test data labels that will be produced by MLP
int Row_Index , Col_Index; 
float Label;

// class constructor just declares / defines attributes
public Classifier()
{
    Digits_Classifier = ANN_MLP.create();

    Layers = new Mat(3,1,CvType.CV_32FC1);
    Training_Samples = new Mat(4,2,CvType.CV_32FC1);
    Training_Responses = new Mat(4,1,CvType.CV_32FC1);
    Test_Samples = new Mat(1,2,CvType.CV_32FC1);
    Test_Responses = new Mat(1,1,CvType.CV_32FC1);
}

public void Start()
{
    // Fill in Layers Mat
    int Layers_Arr[] = new int[]{2,2,1};
    for(int i=0 ; i<3 ; i++)
        Layers.put(i,0,Layers_Arr[i]);

    Row_Index = 0;
    Col_Index = 0;

    // Fill in Training_Samples Mat
    float[] Training_Samples_Arr = new float[]{0,0,0,1,1,0,1,1};
    for(int i=0 ; i<8 ; i++) {
        Training_Samples.put(Row_Index,Col_Index,Training_Samples_Arr[i]);
        Col_Index++;
        if(Col_Index >= 2)
        {
            Row_Index++;
            Col_Index = 0;
        }
    }

    // Fill in Training_Responses Mat
    float[] Training_Responses_Arr = new float[]{0,1,1,0};
    for(int i=0 ; i<4 ; i++)
        Training_Responses.put(i,0,Training_Responses_Arr[i]);

    // Fill in Test_Samples Mat
    double[] Test_Samples_Arr = new double[] {1.0,0.0};
    for(int i=0 ; i<2 ; i++)
        Test_Samples.put(0,i,Test_Samples_Arr[i]);

    Digits_Classifier.setLayerSizes(Layers);

    // The other 2 parameters other than type are Alpha and Beta
    Digits_Classifier.setActivationFunction(ANN_MLP.SIGMOID_SYM, 1.0, 1.0);

    // Term_Criteria object is created with 3 parameters : type , Iterations_Num , Error_Limit
    TermCriteria Digits_Classifier_Term_Criteria = new TermCriteria(TermCriteria.EPS,500,0.001);
    Digits_Classifier.setTermCriteria(Digits_Classifier_Term_Criteria);

    // the other 2 parameters other than type are momentum and weight scales
    Digits_Classifier.setTrainMethod(ANN_MLP.BACKPROP,0.1,0.1);

    Digits_Classifier.train(Training_Samples,Ml.ROW_SAMPLE,Training_Responses);

    // I think this method is the source of problem I face. 
    Label = Digits_Classifier.predict(Test_Samples,Test_Responses,StatModel.RAW_OUTPUT);
}

}

so to summarize this is a 2-2-1 Neural net I use to perform XOR problem. I trained the network by using train method just once I then tried to predict what will the output be for test_sample such as 1 , 0 but the predict method returns 0 although it must be one and another thing is that I provided this example as a training sample what is the problem ?

Note : I have edited older post to be this one to make it more clear enough to understand hoping I can find the problem.

MLP XOR classification problem with predict method

Note : I misused the noun of my object as I named it Digits_Classifier putting in mind I want to classify digits but I started with this simple example to try to understand OpenCV but I faced this problem here :

Does anyone have any idea of why this code produces incorrect classification labels

public class Classifier {

// This is an MLP classifier
// We will use an MLP with structure 2-2-1 to do the task of classifying XOR problem

// MLP Attributes
ANN_MLP Digits_Classifier;
Mat Layers; // A mat contains number of neurons in each layer starting from the input till the output
Mat Training_Samples; // Training data
Mat Training_Responses; // Training data labels
Mat Test_Samples; // Test data 
Mat Test_Responses; // Test data labels that will be produced by MLP
int Row_Index , Col_Index; 
float Label;

// class constructor just declares / defines attributes
public Classifier()
{
    Digits_Classifier = ANN_MLP.create();

    Layers = new Mat(3,1,CvType.CV_32FC1);
    Training_Samples = new Mat(4,2,CvType.CV_32FC1);
    Training_Responses = new Mat(4,1,CvType.CV_32FC1);
    Test_Samples = new Mat(1,2,CvType.CV_32FC1);
    Test_Responses = new Mat(1,1,CvType.CV_32FC1);
}

public void Start()
{
    // Fill in Layers Mat
    int Layers_Arr[] = new int[]{2,2,1};
    for(int i=0 ; i<3 ; i++)
        Layers.put(i,0,Layers_Arr[i]);

    Row_Index = 0;
    Col_Index = 0;

    // Fill in Training_Samples Mat
    float[] Training_Samples_Arr = new float[]{0,0,0,1,1,0,1,1};
    for(int i=0 ; i<8 ; i++) {
        Training_Samples.put(Row_Index,Col_Index,Training_Samples_Arr[i]);
        Col_Index++;
        if(Col_Index >= 2)
        {
            Row_Index++;
            Col_Index = 0;
        }
    }

    // Fill in Training_Responses Mat
    float[] Training_Responses_Arr = new float[]{0,1,1,0};
    for(int i=0 ; i<4 ; i++)
        Training_Responses.put(i,0,Training_Responses_Arr[i]);

    // Fill in Test_Samples Mat
    double[] Test_Samples_Arr = new double[] {1.0,0.0};
    for(int i=0 ; i<2 ; i++)
        Test_Samples.put(0,i,Test_Samples_Arr[i]);

    Digits_Classifier.setLayerSizes(Layers);

    // The other 2 parameters other than type are Alpha and Beta
    Digits_Classifier.setActivationFunction(ANN_MLP.SIGMOID_SYM, 1.0, 1.0);

    // Term_Criteria object is created with 3 parameters : type , Iterations_Num , Error_Limit
    TermCriteria Digits_Classifier_Term_Criteria = new TermCriteria(TermCriteria.EPS,500,0.001);
    Digits_Classifier.setTermCriteria(Digits_Classifier_Term_Criteria);

    // the other 2 parameters other than type are momentum and weight scales
    Digits_Classifier.setTrainMethod(ANN_MLP.BACKPROP,0.1,0.1);

    Digits_Classifier.train(Training_Samples,Ml.ROW_SAMPLE,Training_Responses);

    // I think this method is the source of problem I face. 
    Label = Digits_Classifier.predict(Test_Samples,Test_Responses,StatModel.RAW_OUTPUT);
}

}

  • so So to summarize this is a 2-2-1 Neural net I use to perform XOR problem. I trained the network by using train method just once I then tried to predict what will the output be for test_sample such as 1 (1 , 0 0) but the predict method returns 0 although it must be one and another thing is that I provided this example as a training sample what is the problem ?

  • Another thing I want to ask about is that I defined Layers to be 3x1 Mat i.e. to have 3 layers one of them is the hidden one but when I use function getweights(int Layer_Index) it accepts until Layer_Index = 4 i.e. I have 5 layers so what ?

Note : I have edited older post to be this one to make it more clear enough to understand hoping I can find the problem.problem and solve it.