Loss stuck at the same value with pretrained model

1

hello, i am new in pytorch. Here i am trying to update my existing model with some additional training, but failed to update the loss value. I couldn’t be able to find what is wrong here.

model= AttnVGG_before()
model.load_state_dict(torch.load(’/content/AttStateDict.pth’))

def train(model, train_X):
best_loss = 10000000
for epoch in range(1000):
train_loss = 0
val_loss = 0
adjust_learning_rate(optimizer, epoch)

    model.train()  
    for data in trainloader:
        X, Y = data[0].to(device, non_blocking=True), data[1].to(device, non_blocking=True)
        optimizer.zero_grad()
        outputs = model(X)
        loss = criterion(outputs,Y)
        loss.backward()
        optimizer.step()

        train_loss += loss.item()

    model.eval()  
    with torch.no_grad():
      for data in testloader:
          X, Y = data[0].to(device, non_blocking=True), data[1].to(device, non_blocking=True)
          outputs = model(X)
          loss = criterion(outputs,Y)
          val_loss += loss.item()
    
    print('[Epoch %d] train_loss: %.8f val_loss : %.8f' %
                  (epoch + 1, train_loss/len(trainloader), val_loss/len(testloader)))
    if val_loss < best_loss:
      torch.save(model.state_dict(), 'AttStateDict.pth')
      best_loss = val_loss
      
print('Done Training')

model.to(device)
train(model, trainloader)

torch.save(model.state_dict(), ‘AttentionState.pth’)

Output

[Epoch 1] train_loss: 0.01428753 val_loss : 0.00846462
[Epoch 2] train_loss: 0.01428753 val_loss : 0.00846462
[Epoch 3] train_loss: 0.01428753 val_loss : 0.00846462

2

The first thing I would check in this situation is if the optimizer is defined correctly with the parameters of the model:

optimize = optim.Adam(model.parameters(), ...)

It could be that there are two instances of model and optimizer is defined on one instance, while loss is computed on a different instance.

1 Like
3

Thank you, it worked