I have a net loaded from onnx:
net = cv2.dnn.readNetFromONNX(xxx.onnx)
when i directly do net.forward(), the inference time is around 0.2s
If I set cuda as backend after loading model:
net.setPreferableBackend(cv2.dnn.DNN_BACKEND_CUDA) net.setPreferableTarget(cv2.dnn.DNN_TARGET_CUDA)
The inference time will be around 0.4s
(But this backend setting is working for YOLOv3 readNetFromDarknet, I can see an inference time reduce after setting cuda as backend)
Is it because the model is not supporting cuda backend?