I assume you used the same training code and just swapped the VGG model for the ResNet?
Are you using both pretrained models or are you training from scratch?
Also, did you make sure to change the fc attribute in resnet34 for your output linear layer and the classifier attribute in your vgg16_bn model?