Measure inference time

I have two models that perform the same operation, one uses a library that executes sparse convolution on gpu while the other is a standard pytorch model running on cpu. I would like to analize the inference time to understand if there is an improvement in using the GPU and sparse convolution, what is the best way to do so? On cpu I usually use datetime before and after executing the model on a batch of images but it might not be the best practice.

A simple approach would be to print datetime right before training starts on the GPU and print datetime once again right after training ends. You can also choose to save the datetime objects and determine the delta_t. See https://www.geeksforgeeks.org/python-difference-between-two-dates-in-minutes-using-datetime-timedelta-method/

This is indeed the approach I took with the CPU model even if I’m sure it is not the best practice. The problem with the GPU one although is that multi threading requires synchronization and simply recording the time before and after the execution of the model might not yield the correct result. I am still searching for the correct way to approach this problem. Thank for your response!

You said you want to compare inference time. Inference begins when data enters the forward pass and ends when it exits the forward pass.

def forward(self, x)
    starttime = time.time()
    x = layers(x)
    print(time.time() - starttime)
    
    return x