import torch from torch import nn from torch.autograd import Variable torch.manual_seed(1) net = nn.Sequential( nn.Linear(100, 100), nn.BatchNorm1d(100), nn.Linear(100, 1) ) im = Variable(torch.ones(10, 100)) for _ in range(10): net.train(True) print(net(im)) for _ in range(10): net.train(False) print(net(im))
the first 10 prints output arrays with one value(
-8.6313 * 1e-2), and the second 10 prints output arrays with another value (
-0.1563 * 1e-2).
It is rather strange and confusing.
the first 10 output is the same, is it reasonable? I think the
running_varhave some initial value and then get updated at each step. so the output should be changing.
the second 10 output is the same, it is reasonable since it’s in eval mode. what is confusing is that it is not the same as the last training output. I think if we set the model to evaluation mode, it just freezes
running_var, which should make the second 10 outputs the same as the last training output.
Any ideas? I’m almost driven crazy by this!