1 | initial version |

I am not sure what's the sense of using LBP codes for the MNIST Dataset. It should be totally sufficient to apply a PCA directly. Also note, that mathematically a k-means method yields the same clusters as a PCA projection, there are papers about this (I am on my mobile phone, so I can't look up the title).

So there's also no real need for performing a k-means clustering on the projected samples. What I would try instead is to see, what clusters a Linear Discriminant Analysis yields. Then I would play around with OpenCV's awesome machine learning library and see how a Multi Layer Perceptron performs or how a SVM performs on the raw image data (with different kernels).

Regarding Local Binary Patterns, there's a much simpler (and tested!) implementation in the face recognition code I provide. The OpenCV implementation in cv::FaceRecognizer is the same. I am linking to the original project (libfacerec), because everything is in one file there:

- https://github.com/bytefish/libfacerec/blob/master/src/lbp.cpp

If you need some ideas on how to work with OpenCV machine learning, you can have a look into my Guide to Machine Learning with OpenCV:

- https://github.com/bytefish/opencv/raw/master/machinelearning/doc/machinelearning.pdf

There are also tutorials on using a SVM in the official OpenCV documentation:

- http://docs.opencv.org/doc/tutorials/ml/table_of_content_ml/table_of_content_ml.html

2 | added citation |

~~I am not sure what's the sense of using LBP codes for the MNIST Dataset. It should be totally ~~First try to apply a PCA directly, I think it's sufficient ~~to apply a PCA directly. Also ~~already. Then note, that mathematically a k-means method yields the same ~~clusters ~~centroids as a PCA projection, ~~there are papers about this (I am on my mobile phone, so I can't look up the title). ~~

- Ding C., He X.
*"K-means Clustering via Principal Component Analysis"*in Proceedings of the 21 st International Conference on Machine Learning, Banff, Canada 2004. [[PDF Online Available here](http://ranger.uta.edu/~chqding/papers/KmeansPCA1.pdf)]

So there's ~~also ~~no ~~real ~~need ~~for ~~to performing a k-means clustering on the projected samples. What I would try instead is to see, what clusters a Linear Discriminant Analysis yields. Then I would play around with OpenCV's awesome machine learning library and see how a Multi Layer Perceptron performs or how a SVM performs on the ~~raw ~~image data (with different kernels).

Regarding Local Binary Patterns, there's a much simpler (and tested!) implementation in the face recognition code I provide. The OpenCV implementation in cv::FaceRecognizer is the same. I am linking to the original project (libfacerec), because everything is in one file there:

- https://github.com/bytefish/libfacerec/blob/master/src/lbp.cpp

If you need some ideas on how to work with OpenCV machine learning, you can have a look into my Guide to Machine Learning with OpenCV:

- https://github.com/bytefish/opencv/raw/master/machinelearning/doc/machinelearning.pdf

There are also tutorials on using a SVM in the official OpenCV documentation:

- http://docs.opencv.org/doc/tutorials/ml/table_of_content_ml/table_of_content_ml.html

3 | No.3 Revision |

First try to apply a PCA directly, I think it's sufficient already. Then note, that mathematically a k-means method yields the same centroids as a PCA projection, see:

- Ding C., He X.
*"K-means Clustering via Principal Component Analysis"*in Proceedings of the 21 st International Conference on Machine Learning, Banff, Canada 2004. [[PDF Online Available here](http://ranger.uta.edu/~chqding/papers/KmeansPCA1.pdf)]

So there's no need to performing a k-means clustering on the projected samples. What I would try instead is to see, what clusters a Linear Discriminant Analysis yields. A Linear Discriminant Analysis is available as `cv::LDA`

in the contrib module of OpenCV:

Then I would play around with OpenCV's awesome machine learning library and see how a Multi Layer Perceptron performs or how a SVM performs on the image data (with different kernels).

Regarding Local Binary Patterns, there's a much simpler (and tested!) implementation in the face recognition code I provide. The OpenCV implementation in cv::FaceRecognizer is the same. I am linking to the original project (libfacerec), because everything is in one file there:

- https://github.com/bytefish/libfacerec/blob/master/src/lbp.cpp

If you need some ideas on how to work with OpenCV machine learning, you can have a look into my Guide to Machine Learning with OpenCV:

- https://github.com/bytefish/opencv/raw/master/machinelearning/doc/machinelearning.pdf

There are also tutorials on using a SVM in the official OpenCV documentation:

- http://docs.opencv.org/doc/tutorials/ml/table_of_content_ml/table_of_content_ml.html

4 | No.4 Revision |

First try to apply a PCA directly, I think it's sufficient already. Then note, that mathematically a k-means method yields the same centroids as a PCA projection, see:

- Ding C., He X.
*"K-means Clustering via Principal Component Analysis"*in Proceedings of the 21 st International Conference on Machine Learning, Banff, Canada 2004.~~[[PDF~~[PDF Online Available~~here](http://ranger.uta.edu/~chqding/papers/KmeansPCA1.pdf)]~~here]

So there's no need to performing a k-means clustering on the projected samples. What I would try instead is to see, what clusters a Linear Discriminant Analysis yields. A Linear Discriminant Analysis is available as `cv::LDA`

in the contrib module of OpenCV:

Then I would play around with OpenCV's awesome machine learning library and see how a Multi Layer Perceptron performs or how a SVM performs on the image data (with different kernels).

- https://github.com/bytefish/libfacerec/blob/master/src/lbp.cpp

- https://github.com/bytefish/opencv/raw/master/machinelearning/doc/machinelearning.pdf

There are also tutorials on using a SVM in the official OpenCV documentation:

- http://docs.opencv.org/doc/tutorials/ml/table_of_content_ml/table_of_content_ml.html

5 | No.5 Revision |

First try to apply a PCA directly, I think it's sufficient already. Then note, that mathematically a k-means method yields the same centroids as a PCA projection, see:

- Ding C., He X.
*"K-means Clustering via Principal Component Analysis"*in Proceedings of the 21 st International Conference on Machine Learning, Banff, Canada 2004. [PDF Online Available here]

So there's no need to ~~performing ~~perform a k-means clustering on the projected samples. What I would try instead is to see, what clusters a Linear Discriminant Analysis yields. A Linear Discriminant Analysis is available as `cv::LDA`

in the contrib module of OpenCV:

Then I would play around with OpenCV's awesome machine learning library and see how a Multi Layer Perceptron performs or how a SVM performs on the image data (with different kernels).

- https://github.com/bytefish/libfacerec/blob/master/src/lbp.cpp

- https://github.com/bytefish/opencv/raw/master/machinelearning/doc/machinelearning.pdf

There are also tutorials on using a SVM in the official OpenCV documentation:

- http://docs.opencv.org/doc/tutorials/ml/table_of_content_ml/table_of_content_ml.html

Copyright OpenCV foundation, 2012-2018. Content on this site is licensed under a Creative Commons Attribution Share Alike 3.0 license.