I think the simpler is the following: In order to make a mat of type CV_64F CV_32F or CV_16Uvisualize properly you must normalize it before and then convert it to CV_8U which is the best case for imshow() to show it, thought from my experience imshow() can handle the other types quite nicely as well. In any case the following should do the trick:
cv::Mat depth_src; // your original depth image of type CV_64F, CV_32F or CV_16U