How to detect color during target detection time and put that on the 3D model to be augmented?

edit

Thank you for your answer, that is one good hint.

Actually I am having trouble using the OpenCV package for unity with Vuforia SDK. Considering your answer, after doing what you have proposed, from these colors acquired of different colored regions, i need to make the 2D texture out of it and apply that texture to the 3D model of this object and then augment it.

Here is the link that would demonstrate better of what I want to achieve, maybe I am not explaining it correctly. Here is the video link to it.

I just need the whole right directions for completing this project for my FYP and I am stuck.

Well that doesn't appear to be doing any color detection at all. Just taking the area corresponding to the 3d model and using it straight as the texture. Take a look at 0:52-0:54, you can see where it stretches.

Just take your known shape, (which you know because you have all the line drawings in your library), cut along the lines (virtually) and paste it as your texture.

:o Yes you're right i guess This is in fact a good idea to just get the frame and cut out the target's region and use the same as texture. Here the problem would be to know the location of target and region to cut because it would be different in every frame. But still I would give it a try, Thanks a lot :)