Detect fish species from contour

edit

My goal is to be able to detect fish species from the contour of a fish. The reason for only using the contour is that the water and the glass where the fishes are to be filmed will be more or less dirty. Because of that the background will have light to give a good silhouette. Number one is to detect the size of the fish and that will be easy with a good contour. But I also want to detect what species the fish belong to.
Anyone have any idea how to go about this? Any pointers in the right direction would be welcome!

These pictures is what I have worked with so far.

"Original"

Contours and bounding rect detected