Pytorch and tensorboard logging

I am trying to use pytorch with tensorboard and I run the tensorboard server with the following command:

tensorboard --logdir=./runs/

Now I am just simulating some fake data as follows:

import numpy as np
import time
from torch.utils.tensorboard import SummaryWriter

train_writer = SummaryWriter(log_dir="./runs/train/")

for i in range(100):
    v = np.random.randint(10, 100)
    train_writer.add_scalar("loss", v, i)
    #time.sleep(1)

train_writer.close()

Now if I do not put the sleep method in there, the script finishes and I can see the graph on the tensorboard front end.

I put the sleep in there so that I could see the data arrive in a streaming fashion and be able to see the board updating. However, if I put the sleep in there, no graph ever shows up and all I see is the message that there is no data or it could not find any.

I am not sure if this is a tensorboard issue or if I am doing something wrong in using it from within pytorch.

Hi,

I am not sure how this works but maybe it waits for enough points to be added before sending it to the server? If you run only for 10 iterations and let it run until the end does it work?

So, if I add sleep, it does not matter how many iterations I run but it never updates. In fact, even after the program terminates, the chart is not shown. I have to kill tensorboard and restart it for the graph to show.

Interesting…
So it would be tensorboard not updated even though the data has been written to disk?
@smth will know someone more knowledgeable about this that can find the reason for that ?

@lanpa @orionr are the right folks who might know more about this

Hi, can you try if train_writer.flush() solve you problem?