Identifying and tracking drinks bottles in OpenCV *without* machine learning

edit

Earlier this year Nick Bourdakos posted a series of tweets demoing drinks bottle detection and labelling using IBM's cloud annotation tool (built on top of Tensorflow)

https://mobile.twitter.com/bourdakos1...

I'd be interested in views from experts on this forum as to how close we could get to these results in OpenCV without machine learning.

I conducted a few initial experiments based on this tutorial and found that I could identify and label bottles pretty easily based or the label or bottle colour. However, I was unable to figure out how to extend the bounding box around the whole bottle rather than just the coloured region. I also considered using edge detection and identifying bottle or "not bottle" based on width / height ratio of the bottle's edge contour but due to the bottles being "hand held" it seems difficult to separate the hand to get a solid bottle edge.

If anyone has any thoughts on the best way to achieve this, or even if it is possible at all, I'd be interested to learn more.

Thanks.