Tensor size in ResNet trainting

1

Hello! I’m training a ResNet and i get this error:

out += residual

RuntimeError: The size of tensor a (16) must match the size of tensor b (32) at non-singleton dimension 3

this is my code:

class ResidualBlock(nn.Module):
    expansion = 1
    def __init__(self, in_channels, out_channels, stride=1, downsample=None):
        super(ResidualBlock, self).__init__()
        self.conv1 = conv3x3(in_channels, out_channels, stride)
        self.bn1 = nn.BatchNorm2d(out_channels)
        self.relu = nn.ReLU(inplace=True)
        self.conv2 = conv3x3(out_channels, out_channels)
        self.bn2 = nn.BatchNorm2d(out_channels)
        self.downsample = downsample
        self.shortcut = nn.Sequential()
        if stride != 1 or in_channels != self.expansion * out_channels:
            self.shortcut = nn.Sequential(conv3x3(in_channels, out_channels, stride),
                                          nn.BatchNorm2d(self.expansion * out_channels))

    def forward(self, x):
        residual = x
        out = self.conv1(x)
        out = self.bn1(out)
        out = self.relu(out)
        out = self.conv2(out)
        out = self.bn2(out)
        if self.downsample:
            residual = self.downsample(x)
        out += residual
        out = self.relu(out)
        return out

I don’t understand what’s wrong.

2

Based on the error message it seems that the spatial size of the intermediate activation changed compared to the residual, so you might either want to use a downsampling step or change the architecture. I would also recommend to take a look at the torchvision implementation in case you get stuck.