API/ Library for re-orienting 360 degree camera frame efficiently
I am trying to correct the orientation of a 360 degree camera frame using roll, pitch and yaw estimates coming from my C++ Extended Kalman Filter (EKF) code. I am interested in using libraries and packages such as OpenCV, Nvidia Visionworks or OpenVR. I am working on Ubuntu 16.04. Due to the requirements of the project, I am looking to get it done on GPU.
To give more context, I have a camera that provides 360 degree images which also has a set of IMUs integrated to it. There is an initial frame of reference and EKF calculates roll, pitch and yaw based on this frame of reference. As the camera changes the orientation, I want to use the EKF outputs to correct the orientation of 360 degree video frames coming from the camera.
1) So far I haven't seen any APIs or libraries providing this kind of a function for 360 degree cameras. I am looking for suggestions.
2) If this is not readily available, building a function from scratch using CUDA programming a good approach?
Are you just trying to rotate the image (correct for roll) or something more sophisticated?
I don't have any suggestions for a library that would do it out of the box, but could you just use OpenGL to render to an off screen buffer, adjusting the modelview matrix according to your telemetry data? This should be very fast.