When I use the alexnet to train the Cifar10, I use the following code to set the rand seed:
random.seed(666)
np.random.seed(666)
torch.manual_seed(666)
torch.cuda.manual_seed_all(666)
torch.cuda.manual_seed(666)
torch.backends.cudnn.benchmark = True
torch.backends.cudnn.deterministic = True
And I train it twice, and print the input and loss every iter. But I find the loss of the two different running is different, however, the input is the same. Just as the content in red box.
The code is the same, the input is the same, but why the computed loss is different?
Do someone know how it comes? Thank you very much!!