I've always struggled a bit while trying to compile OpenCV with CUDA on my OSX machine. Basically this was due to the fact that Apple has started since some time to move toward clang rather than sticking to g++, that was the preferred compiler in CUDA: Luckily the latest CUDA SDK (5.5) uses also clang, so now it's much easier to compile everything using this nice compiler.
Here's how I compiled OpenCV 2.4.8 on OSX Mavericks 10.9.1 using Xcode 5.0.2 and CUDA 5.5:
- open CMake to set the project, and to the basic configuration in latest Xcode (I think >= 5) there's no more the gcc compiler, deprecated in favor of clang, so go to the CUDA options of the CMAKE project and change CUDA_HOST_COMPILER to use "/usr/bin/clang".
- Apparently CUDA 5.5 supports only the older libstdc++ library and not the more modern libc++, so update CUDA_NVCC_FLAGS to tell nvcc to tell the native compilar to use this older library. Add "-Xcompiler -stdlib=libstdc++; -Xlinker -stdlib=libstdc++"
- Tell also the C++ compiler that compiles the rest of the library to use libstdc++: show the advanced options of CMAKE and go to CMAKE to add "-stdlib=libstdc++" to both CMAKE_CXX_FLAGS and CMAKE_EXE_LINKER_FLAGS
