An alternative is to segment each number and letter, via contour extraction, and analyse their Hu moments against those you pre-calculate on each reference character. Hu moments have the advantages of being rotation, reflection and scaling invariant, hence the name Hu invariants. This method is computationally cheap (i.e. fast): moments are aggregated measures from the shapes (center of mass, center of inertia, etc) so comparisons are made number-to-number, as opposed to pixel-to-pixel comparisons which are more costly.
In this article the whole recognition process is explained step-by-step, with CV code. In the Bradski-Kaehler openCV book (Chapter 8) there is a discussion about Hu moments for character recognition. Finally there is a library function cvMatchShapes which would extract those moments and compare them internally.
Finally, if you search in internet for "Hu moments character recognition", there are many academical references, but they boil down to the explanations quoted above.