---- I am doing some experiments about regression problem using pytorch. (e.g., input a noised image and output a denoised image).
- In the training epoch, I first execute model.train().
- after each epoch, I do validation, and execute model.eval().
I found the validation loss is normal (consistent with training loss) without calling model.eval(). However, the validation loss becomes much higher when executing model.eval(). So I guessed that this phenomenon may disappear in the test phase. Unfortunately, in the test phase, the performance is still bad when calling model.eval(), while is normal without doing it.
I also tried some other CNN tools, such as matconvnet and tensorflow, they both work fine, and the performance is better when calling ´model.eval()´ in the test phase. (I think it is related to the batchnorm module in my network, I hope for a higher performance when using running average and variance, but the results are opposite)
So anyone can help me on this? Thanks in advance.
Here is my network.
class Net(nn.Module): def __init__(self): self.layers = 17 super(Net, self).__init__() self.layer_m = self.make_h_layers() self.layer_f = nn.Conv2d(64, 1, 3, padding=1) def make_h_layers(self): layer_1 = nn.Sequential(nn.Conv2d(1, 64, 3, padding=1), nn.ReLU(inplace=True)) layer_m = nn.Sequential(nn.Conv2d(64, 64, 3, padding=1), nn.BatchNorm2d(64), nn.ReLU(inplace=True)) layers =  layers.append(layer_1) for i in range(self.layers-2): layers.append(layer_m) return nn.Sequential(*layers) def forward(self, x): x = self.layer_m(x) x = self.layer_f(x) return x