KFOLD Cross Validation Provides extremely unexpected Results?

I’m using transfer learning with k-fold cross validation. k=9 and k folds are being generated using sklearn. The problem i’m facing is that after evaluation of first fold performance decreases drastically. In first fold i get 54% validation accuracy and after that even the training accuracy doesn’t exceed 4% for all the remaining folds.

First guess was that it is probably because of data difference in all folds. So i have run the model multiple times but this behavior still exists. It doesn’t seem to be because of data as new run generates shuffled new data for all folds. I’m completely out of ideas why its happening. Any help is appreciated. Thank You.

def loadModel():
model = sphere20a(classnum=1000).to(device)
model_state = model.state_dict()
for name, param in model.named_parameters():
if (name[:4] == ‘conv’ or name[:4] == ‘relu’) and name[4] is not ‘4’:
param.requires_grad = False

pretrained_state = torch.load("/kaggle/input/kinmodel/sphere20a_20171020.pth")
pretrained_state = {k:v for k,v in pretrained_state.items() if k in model_state and v.size() == model_state[k].size()}

return model

training loop.

for train_index, val_index in kf.split(data):

learn_rate = 0.01
counter = counter + 1
model = loadModel() #load new model everytime and discard old one

I am deleting the old model after every fold completes. using del model

If somebody is up for the challenge of finding the problem i can share the complete notebook.

Could you post the complete code, so that I could have a look?
I’m always up for a challenge! :wink:

Sorry for replying late. I got caught up in some stuff. I have made the notebook public. Here is the link: https://www.kaggle.com/ali14tahir/sphereface-model

I can also share it with you kaggle if you like. I haven’t changed anything since then as I don’t understand what’s going on anymore