Save elapsed time per epoch to txt/csv file?

1

Hello all
I am a beginner in deep learning and pytorch and train my model like code below. I want to ask is it possible to save my elapsed time per epoch in a file . I want to do it because i want to see the max,min,and average elapsed time.

epochs = 5
train_losses=[]

for epochs in range(epochs):
    t0 = time.time()
    model.train()
    for batch_idx, (x,y) in enumerate(loader):
        x,y = x.to(device), y.to(device)
        optimizer.zero_grad()
        outputs = model(x)
        loss = loss_fn(outputs,y)
        loss.backward()
        optimizer.step()
        StopatLossValue()
        
        # program timing
        torch.cuda.synchronize() 
        t1 = time.time()
        eta = t1-t0

        if batch_idx % 100 == 0:
            print('Train Epoch: {} \t Loss: {:.6f} \t Elapsed time :{:.2f} ms'.format(epochs, loss.item(), ((eta)*1000)))
2

Hi,

Your question is not directly affecting PyTorch, so you can save the eta variable in a list with same length of number of epochs and at the end of the loops, write it into a file using python open(file_path, 'w+') method.
There are other ways to do it though.

But something you should consider is that writing to a file is a slow process in comparison to tensor operations, so it is not good idea to write into files within your training or testing workflow, so just save the value in a array and when the loop finished, write to disk.

Bests
Nik

3

thank you for your reply. I have managed to save it