Optimizing pytorch model for single board computers (Jetson nano)

Hi ,
I dont know whether my question is out of scope to this forum so pls bear with me. I am trying to load a resnet34 model which was trained and saved on my laptop (with pytorch) . I copied the the .pkl (pickle) file and .pth (pytorch model) files from my laptop to jetson nano. Now I ran my code which would implement these models . But the code ran very slow . I think the model loading would have taken so much time. It took more than a minute to detect and give me the output. My question is can i use tensorrt to optimize my model? if so how as I already have my .pth file ready , so i think i dont have to retrain my model in jetson nano as that would be tedious , please help!!!

NOTE: what exactly I did was in my laptop collected various environmental sound dataset and converted them into mel-spectogram using librosa and trained those spectograms in my laptop using pytorch after which i got my .pkl and .pth files

You could try to convert your PyTorch model to ONNX and then TensorRT as described in this blog post for WaveGlow.
Also, you could have a look at this repository, which seems to be a bit older by now, so I’m not sure which operations will work and which won’t.

Thank you so much sir @ptrblck I already have my “.pth” file trained in my laptop I just used the same file in jetson nano which was found to be very slow comparatively. Is there any way to optimize only the “.pth” file?

Assuming you are storing the state_dict in the file, I think you would have to recreate the model and follow the export pipeline.

Thank you so much @ptrblck