Ask Your Question

Revision history [back]

onnx model shows slower inference when setting cuda as backend

I have a net loaded from onnx:

net = cv2.dnn.readNetFromONNX(xxx.onnx)

when i directly do net.forward(), the inference time is around 0.2s

If I set cuda as backend after loading model:

net.setPreferableBackend(cv2.dnn.DNN_BACKEND_CUDA) net.setPreferableTarget(cv2.dnn.DNN_TARGET_CUDA)

The inference time will be around 0.4s

(But this backend setting is working for YOLOv3 readNetFromDarknet, I can see an inference time reduce after setting cuda as backend)

Is it because the model is not supporting cuda backend?