---- I am doing some experiments about regression problem using pytorch. (e.g., input a noised image and output a denoised image).
1. In the training epoch, I first execute model.train().
2. after each epoch, I do validation, and execute model.eval().
I found the validation loss is normal (consistent with training loss) without calling model.eval(). However, the validation loss becomes much higher when executing model.eval(). So I guessed that this phenomenon may disappear in the test phase. Unfortunately, in the test phase, the performance is still bad when calling model.eval(), while is normal without doing it.
I also tried some other CNN tools, such as matconvnet and tensorflow, they both work fine, and the performance is better when calling ´model.eval()´ in the test phase. (I think it is related to the batchnorm module in my network, I hope for a higher performance when using running average and variance, but the results are opposite)
So anyone can help me on this? Thanks in advance.
Here is my network.
self.layers = 17
self.layer_m = self.make_h_layers()
self.layer_f = nn.Conv2d(64, 1, 3, padding=1)
layer_1 = nn.Sequential(nn.Conv2d(1, 64, 3, padding=1), nn.ReLU(inplace=True))
layer_m = nn.Sequential(nn.Conv2d(64, 64, 3, padding=1), nn.BatchNorm2d(64), nn.ReLU(inplace=True))
layers = 
for i in range(self.layers-2):
def forward(self, x):
x = self.layer_m(x)
x = self.layer_f(x)