Loaded pytorch model gives different results than originally trained model

Yes, assuming the model does not apply any randomness (e.g. model.eval() will disable dropout layers) and the same data is used. In this case you would expect to see the same output up to the expected mismatches caused by the limited floating point precision (you could also enable deterministic outputs for a performance hit).

A quick search pointed to e.g. this issue which seems to be caused by a difference in the validation set.