 Visualize live graph of lose and accuracy

Is there a simple way to plot the loss and accuracy live during training in pytorch?

1 Like

You could use Visdom for this. Have a look at this example to plot your stats live.
Let me know if you encounter any issues.

1 Like

New to pytorch.
Too much of code, looks complicated.
Unable to get what the actual code is doing.

Used this

import visdom

vis = visdom.Visdom()

#after calculating error per epoch
vis.line(Y=epoch_loss, opts=dict(showlegend=True))

I’m getting this error

Epoch 0/32

train Loss: 2.4588 Acc: 0.5385
Traceback (most recent call last):
File “simpletest.py”, line 207, in
model_ft = train_model(net,criterion,optimizer_ft,exp_lr_scheduler,num_epochs=33)
File “simpletest.py”, line 175, in train_model
vis.line(Y=epoch_loss, opts=dict(showlegend=True))
File “/home/ffffff/.virtualenvs/LearnPytorch/lib/python3.6/site-packages/visdom/init.py”, line 288, in wrapped_f
return f(*args, **kwargs)
File “/home/ffffff/.virtualenvs/LearnPytorch/lib/python3.6/site-packages/visdom/init.py”, line 1171, in line
assert Y.ndim == 1 or Y.ndim == 2, ‘Y should have 1 or 2 dim’
AttributeError: ‘float’ object has no attribute ‘ndim’

Shouldn’t the plot be error per epoch?

what would i use then matplotlib?

Try to pass the values as numpy arrays, e.g.

viz = visdom.Visdom()
epoch_loss = np.array([1.2])
viz.line(epoch_loss)

This did work, found it here https://www.programcreek.com/python/example/92356/visdom.Visdom

import visdom

vis = visdom.Visdom()

loss_window = vis.line(
Y=torch.zeros((1)).cpu(),
X=torch.zeros((1)).cpu(),
opts=dict(xlabel='epoch',ylabel='Loss',title='training loss',legend=['Loss']))

vis.line(X=torch.ones((1,1)).cpu()*epoch,Y=torch.Tensor([epoch_loss]).unsqueeze(0).cpu(),win=loss_window,update='append')

How can I plot two lines in the same window?

You have to concatenate the values as column vectors:

Y = np.linspace(-5, 5, 100)
viz.line(
Y=np.column_stack((Y * Y, np.sqrt(Y + 5))),
X=np.column_stack((Y, Y)),
opts=dict(markers=False),
)
1 Like

I am trying to plot the train loss and validation loss in the same window in Visdom but it’s giving me one line instead of two and also wrong values.

this is the snippet I am using.

criterion = nn.NLLLoss()
optimizer = optim.Adam(model.parameters(), lr=0.001)  # 1e-3
#optimizer = optim.SGD(model.parameters(), lr=0.00001, momentum=0.9)  # 1e-3

# Decay LR by a factor of 0.1 every 4 epochs.
#step size: Period of learning rate decay.
#gamma = Multiplicative factor of learning rate decay. Default: 0.1, should float
scheduler = lr_scheduler.StepLR(optimizer, step_size=2, gamma=0.1)

def train_model(model, criterion, optimizer, scheduler, num_epochs=4):

vis = visdom.Visdom()
loss_window = vis.line(X=numpy.zeros((1 ,)),
Y=numpy.zeros((1)),
opts=dict(xlabel='epoch',
ylabel='Loss',
title='epoch Loss and accuracy',
))
since = time.time()

best_model_wts = copy.deepcopy(model.state_dict())
best_acc = 0.0

for epoch in range(num_epochs):  # loop over the dataset multiple times
print('Epoch {}/{}'.format(epoch, num_epochs - 1))
print('-' * 30)

# Each epoch has a training and validation phase
for phase in ['train', 'valid']:
if phase == 'train':
scheduler.step()

model.train()  # Set model to training mode
else:
model.eval()   # Set model to evaluate mode

train_loss = 0.0
total_train = 0
correct_train = 0

#iterate over data
for t_image, mask, image_paths, target_paths in dataLoaders[phase]:

# get the inputs
t_image = t_image.to(device)

# zeroes the gradient buffers of all parameters

# forward
# track history if only in train
with torch.set_grad_enabled(phase == 'train'):
outputs = model(t_image)
_, predicted = torch.max(outputs.data, 1)
loss = criterion(outputs, mask) # calculate the loss

# backward + optimize only if in training phase
if phase == 'train':
loss.backward() # back propagation
optimizer.step() # update gradients

# accuracy
train_loss += loss.item()
total_train += mask.nelement()  # number of pixel in the batch
correct_train += predicted.eq(mask.data).sum().item() # sum all precited pixel values

epoch_loss = train_loss / len(dataLoaders[phase].dataset)
epoch_acc = (correct_train / total_train)

print('{} Loss: {:.4f} Acc: {:.4f}'.format(
phase, epoch_loss, epoch_acc))

X1 = numpy.ones((1, 1))*epoch
Y1 = numpy.array([epoch_loss ])
Y2 = numpy.array([epoch_loss ])
vis.line(
X=numpy.column_stack((X1, X1)),
Y=numpy.column_stack((Y1, Y2)),
win=loss_window,
update='append')

#deep copy the model
if phase == 'val' and epoch_acc > best_acc:
best_acc = epoch_acc
best_model_wts = copy.deepcopy(model.state_dict())

print()
time_elapsed = time.time() - since
print('Training complete in {:.0f}m {:.0f}s'.format(
time_elapsed // 60, time_elapsed % 60))
print('Best val Acc: {:4f}'.format(best_acc))

torch.save(model.state_dict(), 'train_valid_exp1.pth')
# load best model weights
return model

if __name__=='__main__':
train_model()
1 Like

Currently you are plotting the same values twice (Y1 and Y2 have the same value).
I would rather store the train and val losses in an array and plot them together after both were calculated.
Let me know, if that helps.

1 Like

@ptrblck Could you please give me further explanation or example? Do you mean should I store epoch_loss one time for train and val separately as a numpy array and then plot them?

Yes, since you are currently overwriting it.
Since you are using phase to switch between train and valid, you could store the losses in a dict using the same keys:

losses = {}
for phase in ['train', 'valid']:
...
losses[phase] = epoch_loss
2 Likes

You could try Minetorch , it’s a wrapper of PyTorch which support both Tensorboard and Matplotlib to visualize the loss and accuracy out of box. There’s a mnist sample you could try.

Some visualization

mnist example visualized with matplotlib

mnist example visualized with Tensorboard

2 Likes

Have a look at https://www.comet.ml and https://hyperdash.io. Both platforms provide real time monitoring of training parameters and are easy to use.

1 Like

You can also try Pierogi (Free and Open Source) :

https://github.com/stared/livelossplot worked like a charm for me! Very easy to use compared to some of the alternatives listed here!

@Santhoshnumberone if you are new to pytorch i would suggest, https://medium.com/datadriveninvestor/visualizing-training-and-validation-loss-in-real-time-using-pytorch-and-bokeh-5522401bc9dd

Very less functionalities… After starting from this, what I use is the tensorboard-pytorch module to get it to tensorboard-- https://pytorch.org/docs/stable/tensorboard.html