Essential Matrix from bearing vectors

edit

openMVG has an essential matrix solver using bearing vectors (from SIFT 2D features matches from a spherical panorama) - see robust_essential_spherical.cpp. It works well for retrieving the pose between 2 panoramas. The findEssentialMatrix in OpenCV doesn't support 3D points as such and providing it 2D points from an equirectangular image finds inliers but doesn't find the essential matrix correctly. Is there a similar path of finding the essential matrix (poses) of equirectangular images that doesn't involve converting to cubic format and using the epipolar geometry methods?