# Finetune example doubt at plot curve

@ptrblck: hi!

I’m following the pytorch/finetuning tutoral. Below is some script for plot curve for validation loss

``````ohist = []
shist = []

ohist = [h.cpu().numpy() for h in hist]
shist = [h.cpu().numpy() for h in scratch_hist]

plt.title("Validation Accuracy vs. Number of Training Epochs")
plt.xlabel("Training Epochs")
plt.ylabel("Validation Accuracy")
plt.plot(range(1,num_epochs+1),ohist,label="Pretrained")
plt.plot(range(1,num_epochs+1),shist,label="Scratch")
plt.ylim((0,1.))
plt.xticks(np.arange(1, num_epochs+1, 1.0))
plt.legend()
plt.show()

``````

could help me to understand this, as i didn’t get what’s going here: `plt.plot(range(1,num_epochs+1),ohist,label="Pretrained")`,

The code creates a plot for the validation accuracy of the fine-tuned model (`ohist`) and the model trained from scratch (`shist`).
Both accuracies will be visualized in a plot where the epochs are given in the x-axis and the accuracies in the y-axis.

PS: I’m not a fan of tagging certain people, as this might discourage others to post an answer.

1 Like

Sure. Won’t tag again. So, here’s where `hist` came from:

``````model_ft, hist = train_model(model_ft, dataloaders_dict, criterion, optimizer_ft, num_epochs=num_epochs, is_inception=(model_name=="inception"))
``````

and the `train_model` function:

``````def train_model(model, dataloaders, criterion, optimizer, num_epochs, is_inception=False):
since = time.time()

val_acc_history = []

best_model_wts = copy.deepcopy(model.state_dict())
best_acc = 0.0

for epoch in range(num_epochs):
print('Epoch {}/{}'.format(epoch, num_epochs - 1))
print('-' * 10)

# Each epoch has a training and validation phase
for phase in ['train', 'val']:
if phase == 'train':
model.train()  # Set model to training mode
else:
model.eval()   # Set model to evaluate mode

running_loss = 0.0
running_corrects = 0

# Iterate over data.
inputs = inputs.to(device)
labels = labels.to(device)

# forward
# track history if only in train
# Get model outputs and calculate loss
# Special case for inception because in training it has an auxiliary output. In train
#   mode we calculate the loss by summing the final output and the auxiliary output
#   but in testing we only consider the final output.
if is_inception and phase == 'train':
# From https://discuss.pytorch.org/t/how-to-optimize-inception-model-with-auxiliary-classifiers/7958
outputs, aux_outputs = model(inputs)
loss1 = criterion(outputs, labels)
loss2 = criterion(aux_outputs, labels)
loss = loss1 + 0.4*loss2
else:
outputs = model(inputs)
loss = criterion(outputs, labels)

_, preds = torch.max(outputs, 1)

# backward + optimize only if in training phase
if phase == 'train':
loss.backward()
optimizer.step()

# statistics
running_loss += loss.item() * inputs.size(0)
running_corrects += torch.sum(preds == labels.data)

print('{} Loss: {:.4f} Acc: {:.4f}'.format(phase, epoch_loss, epoch_acc))

# deep copy the model
if phase == 'val' and epoch_acc > best_acc:
best_acc = epoch_acc
best_model_wts = copy.deepcopy(model.state_dict())
if phase == 'val':
val_acc_history.append(epoch_acc)

print()

time_elapsed = time.time() - since
print('Training complete in {:.0f}m {:.0f}s'.format(time_elapsed // 60, time_elapsed % 60))
print('Best val Acc: {:4f}'.format(best_acc))

return model, val_acc_history
``````

That plots only the validation accuracy. But I want plots such that it has to plot

1. Training Acc/Loss Vs. Epochs
2. Validation Acc/Loss Vs.Epochs

any suggestions?

You would have to store `epoch_acc` in another list for the training phase:

``````def train_model(...):
train_acc_history = []
val_acc_history = []
...

for epoch in range(num_epochs):
for phase in ['train', 'val']:
...

if phase == 'val':
val_acc_history.append(epoch_acc)
elif phase == 'train':
train_acc_history.append(epoch_acc)
...

return model, val_acc_history, train_acc_history
``````
1 Like

Thanks. I tried by adding these lines:

``````            if phase == 'val' and epoch_acc > best_acc:
best_acc = epoch_acc
best_model_wts = copy.deepcopy(model.state_dict())
if phase == 'val':
val_acc_history.append(epoch_acc)
val_acc_history.append(epoch_loss)
elif phase == 'train':
train_acc_history.append(epoch_acc)
train_acc_history.append(epoch_loss)

....
....
model_ft, hist = train_model(model_ft, dataloaders_dict, criterion, optimizer_ft, num_epochs=num_epochs, is_inception=(model_name=="inception"))

ohist = []
#shist = []
ohist = [h.cpu.numpy() for h in hist]
#shist = [h.cpu().numpy() for h in train_model.epoch_loss]

plt.title("Loss&Acc Vs. Number of Training Epochs")
plt.xlabel(" Epochs")
plt.ylabel("Accuracy & Loss")
plt.plot(range(1,num_epochs+1),ohist,label="Training Accuracy")
plt.plot(range(1,num_epochs+1),ohist,label="Training Loss")
plt.plot(range(1,num_epochs+1),ohist,label="Validation Accuracy")
plt.plot(range(1,num_epochs+1),ohist,label="Validation Loss")
plt.ylim((0,1.))
plt.xticks(np.arange(1, num_epochs+1, 1.0))
plt.legend()
plt.savefig('plots.png')
plt.show()

``````

but it throws an error:

``````Traceback (most recent call last):
File "ftune.py", line 284, in <module>
model_ft, hist = train_model(model_ft, dataloaders_dict, criterion, optimizer_ft, num_epochs=num_epochs, is_inception=(model_name=="inception"))
ValueError: too many values to unpack (expected 2)
``````

I tried also TensorboardX, like these snippets:

``````from tensorboardX import SummaryWriter
writer = SummaryWriter('runs')
...
...
def train_model(train_loader, model, criterion, optimizer, epoch):
if i % args.print_freq == 0:
print('Epoch: [{0}][{1}/{2}]\t'
'Time {batch_time.val:.3f} ({batch_time.avg:.3f})\t'
'Data {data_time.val:.3f} ({data_time.avg:.3f})\t'
'Loss {loss.val:.4f} ({loss.avg:.4f})\t'
'Prec@1 {top1.val:.3f} ({top1.avg:.3f})\t'
'Prec@5 {top5.val:.3f} ({top5.avg:.3f})'.format(
data_time=data_time, loss=losses, top1=top1, top5=top5))

if i % args.print_freq == 0:
print('Test: [{0}/{1}]\t'
'Time {batch_time.val:.3f} ({batch_time.avg:.3f})\t'
'Loss {loss.val:.4f} ({loss.avg:.4f})\t'
'Prec@1 {top1.val:.3f} ({top1.avg:.3f})\t'
'Prec@5 {top5.val:.3f} ({top5.avg:.3f})'.format(
top1=top1, top5=top5))

``````

but in `validate` function it throws an error like epoch is not defined etc.

could you share some snippets for TensorboardX?

The first error is thrown, since you are now returning three objects/values:

``````return model, val_acc_history, train_acc_history
``````

The second is thrown, because `epoch` is indeed not defined in `validate`, so you could pass it as an argument to the function.

1 Like

Can I export validation plot data to a a csv file?

Any example?

This should be possible using `pandas`.
You would have to create a `pd.DataFrame` containing your validation values and save it using `df.to_csv`.

1 Like

Hi, I am running into the same problem as I am a beginner. I was wondering how do you change the code so it expects all three objects. Even though you include all three objects at the return statement why does it still say it expects only 2.

I’m not sure I understand the issue correctly.
Could you post the error message or explain your use case a bit?
Which function expects two arguments?

I am just having trouble creating a graph for my loss values. The tutorial (https://pytorch.org/tutorials/beginner/finetuning_torchvision_models_tutorial.html) only shows how to create the graph for validation accuracy, however I wanted to plot the validation loss as well. The error I am getting occurs after I try to make a list for a variable called val_loss_history, similar to how they wrote val_acc_history.

Apologies in advance if this is confusing.

Since you are returning 3 values in `train_model`, you would also have to assign them:

``````model_ft, hist, losses = train_model(model_ft, dataloaders_dict, criterion, optimizer_ft, num_epochs=num_epochs, is_inception=(model_name=="inception"))
``````

PS: you can add code snippets by wrapping them in three backticks ```

Thank you, I was also wondering how I could save the trained model after every epoch. I am currently saving it like so:

, but that saves the entire model.

That’s the right approach of saving the model’s `state_dict`.
Why is it unexpected to save the entire model?

I want to save it using checkpoints as my best val accuracy is at epoch 10, and I trained to epoch 14. I was looking at this discussion on how to do it How resume the saved trained model at specific epoch, however I don’t know how to change my line (listed below) such that it saves the entire model using those checkpoints at every epoch.

torch.save(model_ft.state_dict(), ‘C:\Users\Niharika\Desktop\CavitiesCNN\results6\model.pth’)
torch.save(optimizer.state_dict(), ‘C:\Users\Niharika\Desktop\CavitiesCNN\results6\optimizer.pth’).

If you want to store a checkpoint for every epoch, you could add the epoch counter to the name:

``````torch.save(model_ft.state_dict(), 'C:\Users\Niharika\Desktop\CavitiesCNN\results6\model_epoch{}.pth'.format(epoch))
``````