Optimizing pytorch model for single board computers (Jetson nano)

1

Hi ,
I dont know whether my question is out of scope to this forum so pls bear with me. I am trying to load a resnet34 model which was trained and saved on my laptop (with pytorch) . I copied the the .pkl (pickle) file and .pth (pytorch model) files from my laptop to jetson nano. Now I ran my code which would implement these models . But the code ran very slow . I think the model loading would have taken so much time. It took more than a minute to detect and give me the output. My question is can i use tensorrt to optimize my model? if so how as I already have my .pth file ready , so i think i dont have to retrain my model in jetson nano as that would be tedious , please help!!!

NOTE: what exactly I did was in my laptop collected various environmental sound dataset and converted them into mel-spectogram using librosa and trained those spectograms in my laptop using pytorch after which i got my .pkl and .pth files

2

You could try to convert your PyTorch model to ONNX and then TensorRT as described in this blog post for WaveGlow.
Also, you could have a look at this repository, which seems to be a bit older by now, so I’m not sure which operations will work and which won’t.

3

Thank you so much sir @ptrblck I already have my “.pth” file trained in my laptop I just used the same file in jetson nano which was found to be very slow comparatively. Is there any way to optimize only the “.pth” file?

4

Assuming you are storing the state_dict in the file, I think you would have to recreate the model and follow the export pipeline.

5

Thank you so much @ptrblck