How to extract pixels from a rotated box2d?

edit

A first step is to stop using the old outdated C-API and move on to the new C++ functionality, in your case the minAreaRect function.

What you have to define is the rotation matrix between your box and a horizontal box. More information can be found in in this great answer of Philip Wagner (and could possibly be found with a simple google search - was my first hit).

The code needed to do what you need is something like this template:

// rect is the RotatedRect (I got it from a contour...)
RotatedRect rect;
// matrices we'll use
Mat M, rotated, cropped;
// get angle and size from the bounding box
float angle = rect.angle;
Size rect_size = rect.size;
// thanks to http://felix.abecassis.me/2011/10/opencv-rotation-deskewing/
if (rect.angle < -45.) {
   angle += 90.0;
   swap(rect_size.width, rect_size.height);
}
// get the rotation matrix
M = getRotationMatrix2D(rect.center, angle, 1.0);
// perform the affine transformation
warpAffine(src, rotated, M, src.size(), INTER_CUBIC);
// crop the resulting image
getRectSubPix(rotated, rect_size, rect.center, cropped);