Should I use focal length from camera matrix or new camera matrix for depth estimation?

edit

I use initUndistortRectifyMap using camera matrix and new camera matrix to get output maps. New camera matrix was obtained using estimateNewCameraMatrixForUndistortRectify and passing a non-zero balance value. The new camera matrix as compared to camera matrix has different focal length.

Now the question is, if I get disparity values, first using remap with the output maps and then using a stereo disparity algorithm, to get the real depth what focal length value should I use? The one from old camera matrix or from the new camera matrix?