Pytorch and tensorboard logging

vision
1

I am trying to use pytorch with tensorboard and I run the tensorboard server with the following command:

tensorboard --logdir=./runs/

Now I am just simulating some fake data as follows:

import numpy as np
import time
from torch.utils.tensorboard import SummaryWriter

train_writer = SummaryWriter(log_dir="./runs/train/")

for i in range(100):
    v = np.random.randint(10, 100)
    train_writer.add_scalar("loss", v, i)
    #time.sleep(1)

train_writer.close()

Now if I do not put the sleep method in there, the script finishes and I can see the graph on the tensorboard front end.

I put the sleep in there so that I could see the data arrive in a streaming fashion and be able to see the board updating. However, if I put the sleep in there, no graph ever shows up and all I see is the message that there is no data or it could not find any.

I am not sure if this is a tensorboard issue or if I am doing something wrong in using it from within pytorch.

2

Hi,

I am not sure how this works but maybe it waits for enough points to be added before sending it to the server? If you run only for 10 iterations and let it run until the end does it work?

3

So, if I add sleep, it does not matter how many iterations I run but it never updates. In fact, even after the program terminates, the chart is not shown. I have to kill tensorboard and restart it for the graph to show.

4

Interesting…
So it would be tensorboard not updated even though the data has been written to disk?
@smth will know someone more knowledgeable about this that can find the reason for that ?

5

@lanpa @orionr are the right folks who might know more about this

1 Like
6

Hi, can you try if train_writer.flush() solve you problem?

7

@Luca_Pamparana are you running TensorBoard without TensorFlow being installed? If so, there was a bug where TB wouldn’t update even if there were updates to an event file. This has been fixed on nightly, so try pip install tb-nightly and see if that solves the issue.

1 Like
8

Thank you. That was it!