Draw onto CV image with OpenGL commands?
I would like to draw some simple augmented reality style 3d graphics over a video which is being read and processed in real time with OpenCV (from a robot camera). I've tried a couple of options but none seem very nice:
- convert each video frame from CV image to GL texture, render it with GL in a GL context, and draw on it with GL (is very slow due to the conversions);
- don't use GL, use CV commands to draw lines and shapes on the CV image. This has many problems with offscreen objects and projections of objects behind the pinhole camera, and if I work to remove them it will feel like re-writing a lot of GL which seems dumb. Also it limits to simple line drawing and I want to render some nicely lit and shaded GL graphics into the frame to look prettier.
I'm surprised as I thought this would be a really standard task for AR-style applications but I'm having trouble finding any nice libraries to make the link for me. I think what I really want is a library that allows me to draw onto the CV image /as a GL context/, so that the GL commands get reinterpreted into actions on the CV arrays rather than as commands send to the graphics card. Does such a thing exist anywhere? Surely there is some standard way to do AR with these two libraries playing nicely together?
There is the data member in Mat which can be used in the glTextImage2D function, so you don't need to convert the image. There is also GL_BGR and GL_BGRA defines for using the OpenCV format.
opencv focusses on computer-vision, not on rendering (your problem)
there are e.g. advanced examples here, but don't expect much out-of-the-box.