Issues predicting variable length input with pre-trained model that was trained with padded values


I am a newbie to pytroch, I trained a model that takes in ll5 input and outputs l*l , I used a resnet to train it and during training I zero-padded the input to have 200 x 200x5 and labels as 200x200, and evaluated the method with a custom function.
But if I want to make another program that load its weight and predict , How should I do it?
I tried by not padding the input data and the label, also made the output shape to be same as length of input and evaluated it, the same way and on the same validation data as before. I found is that the accuracy dropped to half.

Can anyone explain what is the issue and how to predict the variable lengths of data using a pre-trained model. The data I am predicting might have any value even greater than 200