The training and validation

autograd
1

have a minor concern regarding the training and validation phase. Assuming that the loss decreases for N iterations. However, at iteration N+1, the loss increases. Should I use the model parameters from iteration N or N+1 before continuing the training and validation phase.

if val_losses[-1] < best_val_loss:
    best_val_loss = val_losses[-1]
    counter=0
    best_model_wts = copy.deepcopy(model.state_dict())
else:
    counter+=1
    print('Validation loss has not improved since: {:.3f}..'.format(best_val_loss), 'Count: ', str(counter))
    if counter >= early_stop_counter:
        print('Early Stopping Now!!!!')
        model.load_state_dict(best_model_wts)
        break