accurate method to calculate framerate of input stream
Related to the latest questions of mine regarding kinect one and opencv I have a loop where I am transforming the images obtained from kinect to opencv matrices and then I am showing these matrices, the procedure is quite similar to the way we are using with the VideoCapture()
class. A snippet code can be seen below:
while( 1 ){
// Frame
// get color image
IColorFrame* pColorFrame = nullptr;
hResult = pColorReader->AcquireLatestFrame( &pColorFrame );
if( SUCCEEDED( hResult ) ){
hResult = pColorFrame->CopyConvertedFrameDataToArray( bufferSizeColor, reinterpret_cast<BYTE*>( colorBufferMat.data ), ColorImageFormat::ColorImageFormat_Bgra );
// resize for visual reasons
if( SUCCEEDED( hResult ) ){
cv::resize( colorBufferMat, colorMat, cv::Size(), 0.5, 0.5 );
}
}
// get depth image
// // method 1
// IDepthFrame* pDepthFrame = nullptr;
// hResult = pDepthReader->AcquireLatestFrame( &pDepthFrame );
// if( SUCCEEDED( hResult ) ){
// hResult = pDepthFrame->AccessUnderlyingBuffer( &bufferSizeDepth, reinterpret_cast<UINT16**>( &depthBufferMat.data ) );
// if( SUCCEEDED( hResult ) ){
// depthBufferMat.convertTo( depthMat, CV_8U, 0.05f/*-255.0f / 8000.0f, 255.0f*/ );
// }
// }
// method 2
IDepthFrame* pDepthFrame = nullptr;
hResult = pDepthReader->AcquireLatestFrame( &pDepthFrame );
if( SUCCEEDED( hResult ) ){
unsigned int bufferSize = 0;
unsigned short* buffer = nullptr;
hResult = pDepthFrame->AccessUnderlyingBuffer( &bufferSize, &buffer );
if( SUCCEEDED( hResult ) ){
for( int y = 0; y < depthHeight; y++ ){
for( int x = 0; x < depthWidth; x++ ){
unsigned int index = y * depthWidth + x;
BYTE intensity = buffer[ index ] == 0 || buffer[ index ] > 4095/*2047*/ ? 0 : 255 - (BYTE)(((float)buffer[ index ] / 4095.0f/*2047.0f*/) * 255.0f);
depthMat.at<unsigned char>( y, x ) = intensity;
}
}
}
// memcpy(depth_im.data, buffer, depthHeight*depthWidth*sizeof(UINT16)); // creating an opencv image from the raw data in buffer above with the raw data
}
// get a colormap of the depth image
applyColorMap(depthMat, colorDepthMat, COLORMAP_JET);
// get infrared image
IInfraredFrame* pInfraredFrame = nullptr;
hResult = pInfraredReader->AcquireLatestFrame( &pInfraredFrame );
if( SUCCEEDED( hResult ) ){
unsigned int bufferSize = 0;
unsigned short* buffer = nullptr;
hResult/*IR*/ = pInfraredFrame->AccessUnderlyingBuffer( &bufferSize, &buffer );
if( SUCCEEDED( hResult ) ){
for( int y = 0; y < irHeight; y++ ){
for( int x = 0; x < irWidth; x++ ){
unsigned int index = y * irWidth + x;
infraredMat.at<unsigned char>( y, x ) = buffer[ index ] >> 8;
}
}
}
}
SafeRelease( pColorFrame );
SafeRelease( pDepthFrame );
SafeRelease( pInfraredFrame );
cv::imshow( "Color", colorMat );
// raw size
// cv::imshow( "Color", colorBufferMat );
cv::imshow( "Depth", depthMat );
cv::imshow( "Color Depth", colorDepthMat );
cv::imshow( "Infrared", infraredMat );
int key = waitKey(1);
if( key == VK_ESCAPE ){
break;
}
}
the question now is, is there any way to accurate measure the framerate that I get from kinect? I guess it does not have to do strictly related to kinect it could be any normal camera. Searching around I could something clear. Officially it says that kinect supports 30 fps so I would like to see if I get something close to that.
Update
#include <iostream>
#include <time.h>
#include <opencv2/opencv.hpp>
using namespace std;
using namespace cv;
int main()
{
VideoCapture cap1(0);
VideoCapture cap2(1);
// check if video is loaded successfuly
if(!cap1.isOpened() || !cap2.isOpened())
{
cout << "Problem opening streams!" << endl;
return -1;
}
Mat frame1, frame2;
VideoWriter rec1("test.avi", -1/*CV_FOURCC('M','J','P','G')*/, 25, Size(640, 480), true); // setting cv_fourcc manually does not work in windows, also do not know why
VideoWriter rec2("test2.avi", -1/*CV_FOURCC('D','I','V','X')*/, 25, Size(640, 480), true);
if ( !rec1.isOpened() ) //if not ...
Take a look at @pklab's answer here
@LorenaGdL thanks for the link. It seems to be what I wanted, how the heck I missed it from the seach. Thanks again ;-).
Always welcome ;)
@LorenaGdL I am see that using the code you pointed out in the link above I get a constant frame rate. However, when I introduce a
videowriter
in order to save the images in a video the frame rate constantly drops I stopped it at 6 fps, I am sure that if I leave it will go further down. So, I was wondering if you have noticed any similar behavior because I think that it is a bug invideowriter()
or something. I would expect to have a short range drop in the fps but not a continues one.I don't recall measuring times in code including videowriter. If you post some minimal code to reproduce the issue, I'll check later
ok try the code in the update section, for some reason in the beginning before recording Measure time decreases constantly leading to fps increment and after recording (press space for that) Measure time increases constantly leading to less fps. I do not know what is happening, most likely I miss something in the code but I am not sure. Can you have a look and tell me what happens to your computer.
In the beggining there is a constant decrease of processing times, though I expected so: structures need to be initialized, and the computer is general need to start new processes, new threads and so on. After some few frames the fps is stable, in 30fps in my case. After started the recording, fps is still stable (slightly increased times, but still 30 fps). I let it run for a while, and never got less than 30 fps. So I don't know what's happening in your case. I tested with OpenCV 2.4 and 3.0 in Win7 x64, Visual Studio 2013. There is this issue that points to a similar behaviour, though for Python + Mac. Recommendation is to build with FFMPEG, I don't know if you're using it (I am)
@LorenaGdL how to build opencv with FFMPEG in windows because it also seems to be dependent on the codec that I am using. At the cmake phase the FFMEG flag was on before I start the compilation but later I noticed that I couldn't record anything since it couldn't find any codec. After that I installed some codecs from K-lite codecs mega pack and some seems to work some others not. I do not really know what is happening with the ffmpeg in windows, actually is there any guide that describes that for windows because from what I have seen the most guides for windows are missing some information. By the way thanks for your time.
I actually facing exactly this problem described here
I know very little about this kind of video issues. Maybe @pklab (sorry if I bother mentioning you) can help you more, I've seen several posts from him about related issues. I had K-lite codecs installed much before I even started using OpenCV, and Cmake always recognizes FFMPEG as installed in my Windows7 when building. So I don't think I can be of any help