YoloV5x model prediction time


We are using YoloV5x model for object detection. In out setup we have Geforce GT730 GPU and we have build the torch(V1.10) and torchvision (V0.11) from source. The model is able to detect the CUDA and is working. When we check the model prediction time, the time taken from the prediction is high.

In our application we will be using 4 cameras for object detection. When we tested the YoloV5 model with 2 cameras, the time taken is approximately 1.6-1.8 sec with imgsz size of 640.

After training the model we have saved the model with .pt extension.

Is there any way where we can increase the model prediction speed? Also the CPU usage is 100% when we run the model with 2 cameras.


You could profile your application with the native PyTorch profiler or e.g. Nsight Systems and could try to narrow down where the bottleneck is. Once isolated the improvements depend on the actual root cause. For general recommendation check the performance guide.

@ptrblck , can you suggest ways of optimizing model inference , I am using yolov5 models with input size of 1920 px but inference time took around 1000ms per image. Appreciate your response. Thanks.

Same as before: try to profile your workload to narrow down the bottleneck as optimizing code without actually improving the bottleneck might not yield the expected gains.