RuntimeError: running_mean should contain 64 elements not 128

1

self.conv1 = nn.Conv2d(1, 64, (1, 200), (1, 1), (0, 0))
self.batchnorm1 = nn.BatchNorm2d(64)

and, I try,

h = F.relu(self.batchnorm1(self.conv1(h)))

so, error occur as follow,

RuntimeError: running_mean should contain 64 elements not 128

I can’t understand why this error occur.

shape of conv1 is torch.Size([128, 64, 20, 1]) so, batch size is 128.

Please help.

2

Your code is running using an input of [1, 1, 200, 200]:

conv1 = nn.Conv2d(1, 64, (1, 200), (1, 1), (0, 0))
batchnorm1 = nn.BatchNorm2d(64)

h = torch.randn(1, 1, 200, 200)
h = F.relu(batchnorm1(conv1(h)))

Could you check, if you are using batchnorm1 somewhere else in your model?
Maybe you have a typo somewhere.

3

Thank you for your reply !

I create model like this.

# conv layer 1
self.conv1 = nn.Conv2d(1, 64, (1, 200), (1, 1), (0, 0))
self.batchnorm1 = nn.BatchNorm2d(64)

and use as follows,

# x --> input shape (128, 200, 20, 1). It seems 200*20 pixel and 1ch image with minibatch 128.
h = x.view(-1, 1, 200, 20)
h = F.tanh(self.batchnorm1(self.conv1(h)))

And I try this one.

# conv layer 1
self.conv1 = nn.Conv2d(1, 128, (1, 200), (1, 1), (0, 0))
self.batchnorm1 = nn.BatchNorm2d(128)

It looks like good, and doesnt occur any erros. I dont know why former one doesnt work…

4

I am getting the same error. My entire code is taken from https://github.com/hezhangsprinter/DCPDN/blob/master/dehaze22.py and I am getting the error from line 193, I have defined it as:
My layer 1 has:
block = nn.Sequential()
block.add_module(name, nn.Conv2d(in_c,out_c, 4, 2, 1, bias=False))
block.add_module(name, nn.BatchNorm2d(out_c))

When I printed the size of out1 I am getting 1,64,128,128.
I am not sure why I am getting the error from BatchNorm2d

5

I assume you are getting this error in the G model?
If so, could you post the code you are using to initialize it and the shape of the input so that I could debug it?

1 Like
7

Thank you. I fixed the bug. The input was 256*256. Thank you for the response though. In reference to the code, I have another issue which you have addressed Error: Expected more than 1 value per channel when training. I do have a batch_size of 1 . In that case, the BatchNorm2d is not valid. Then, why I am not getting the errors earlier. Like I am only getting the error ValueError: Expected more than 1 value per channel when training, got input size [1, 512, 1, 1] and not when my tensor size is [1, 64, 128, 128] . [line 199 on the code] Upon putting bn=False, the code runs.

1 Like
8

BatchNorm layers compute the running estimates per channel, i.e. the mean and standard deviation would be 512-dimensional tensors. Since you have only one sample in the batch and also only one “pixel” the running estimates cannot be calculated. In that case you could remove the BatchNorm layer and use e.g. InstanceNorm instead.

9

InstanceNorm2d gives the same error.

10

While InstanceNorm2d should work, I’m not sure if the usage would make sense at all, since you’ll get an all zero output tensor if you just have a single pixel.

norm = nn.InstanceNorm2d(512)
norm(torch.randn(1, 512, 1, 1))
11

I think in the functional.py it is calling the batch_norm() which is giving the error.
File “/anaconda3/envs/cyclegan/lib/python3.6/site-packages/torch/nn/functional.py”, line 1293, in _instance_norm
training=use_input_stats, momentum=momentum, eps=eps)
File “/anaconda3/envs/cyclegan/lib/python3.6/site-packages/torch/nn/functional.py”, line 1251, in batch_norm
raise ValueError(‘Expected more than 1 value per channel when training, got input size {}’.format(size))
ValueError: Expected more than 1 value per channel when training, got input size [1, 512, 1, 1]