Hi. I am getting an output by using anomaly detection model which is unsupervised learning model. When I control with time.time(), model inference is looking fast, but when I would like to get image score from model output by using torch.max() or something, fps is decreasing suddenly. What is cause of that. I think this state relevant to autograd (or grad_mode.py call) by pytorch or like this but I cannot ensure. I do not have knowledge enough. Help me please.
Assuming your are using your GPU for the model execution, using host timers requires a device synchronization since CUDA operations are executed asynchronously.
Thank you for your reply.