Plotting loss curve


#1

I am trying to plot a loss curve by each epoch, but I’m not sure how to do that. I can do it for 1 epoch using the following method:

def train(model, num_epoch):

    for epoch in range(num_epoch):
        running_loss = 0.0
        loss_values = []
        for i, data in enumerate(trainloader, 0):
            images, labels, bbox = data
            
            images = Variable(images).to(device)
            labels = Variable(labels).to(device)
                        
            optimizer.zero_grad()
            outputs = model(images)#, th_images)
            loss = criterion(outputs, labels.to(device))
            loss.backward()
            optimizer.step()
            
            running_loss += loss.item()
            
            loss_values.append(running_loss)
            
            if i % 10 == 9:    # print every 2000 mini-batches
                
                print('[%d, %5d] loss: %.3f' %
                      (epoch + 1, i + 1, running_loss / 10))
                running_loss = 0.0
       
    print('Finished Training Trainset')
            
    plt.plot(np.array(loss_values), 'r')

model = train(vgg16, 2)

However, I get the following plot, which doesn’t seem correct:

image

Any advice would be grealty appreciated.


#2

Currently you are accumulating the batch loss in running_loss. If you just would like to plot the loss for each epoch, divide the running_loss by the number of batches and append it to loss_values in each epoch.
Note, that this might give you a slightly biased loss if the last batch is smaller than the others, so let me know if you need the exact loss.


#3

Thank you for your reply. I will probably need an exact loss when I use the full dataset, but I’m only using a small portion of it, so probably don’t require that right now.

If you could tell me how to get exact values, then that would be great for when I use the full dataset.

I’m just a bit coursed about ‘append it to loss_values in each epoch’. Based on the image I posted, the x axis is not the epochs. I believe it is using the optimizer.step values instead.


#4

You could use the ImageNet example or the following manual approach:

for epoch in range(num_epochs):
    running_loss = 0.0
    for i, data in enumerate(trainloader, 0):
        running_loss =+ loss.item() * images.size(0)

    loss_values.append(running_loss / len(train_dataset))

plt.plot(loss_values)

This code would plot a single loss value for each epoch. Would that work?


#5

Thank you for your reply. I can already plot the loss curve for one epoch, but I was looking to plot the loss for a number of epochs. Using the manual approach you suggested above, this is the loss curve generated:

image

What I would like to do is something like this:

I hope that clears up what I wanting to do.


#6

The code I’ve posted should plot a single loss values for each epoch.
I assume you let your model train for 25 epochs, is that correct?

If so, the plots should show basically the same with the difference that the second plot shows the train and validation loss for each epoch.


#7

I am attempting to do only 2 epochs according to the line model = train(vgg16, 2). I am unsure where the 25 on the x axis is coming from. I believe it may have something to do with the optimist.step? Also, my trainset is 100 images and my batch size in the trainloader is 4, so perhaps it maybe something to with that instead?