# RuntimeError: output shape doesn't match broadcast shape on weight loading

**URL:** <https://discuss.pytorch.org/t/runtimeerror-output-shape-doesnt-match-broadcast-shape-on-weight-loading/118809>\
**Category:** vision\
**Created:** [April 20, 2021, 12:24pm UTC](https://discuss.pytorch.org/t/runtimeerror-output-shape-doesnt-match-broadcast-shape-on-weight-loading/118809 "2021-04-20T12:24:21Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![cpeters](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/c/e274bd/32.png) [@cpeters](https://discuss.pytorch.org/u/cpeters)\
**Post date:** [April 20, 2021, 12:24pm UTC](https://discuss.pytorch.org/t/runtimeerror-output-shape-doesnt-match-broadcast-shape-on-weight-loading/118809/1 "2021-04-20T12:24:21Z")

</div>

I’ve been trying to convert the vgg16 model trained on [the places365 dataset](https://github.com/CSAILVision/places365) from caffe to pytorch, and eventually tried [MMdnn](https://github.com/microsoft/MMdnn), which successfully read the caffe files, converted them to intermediate representations, and then to python code. However, when it tries to do the last step (load the model and weights together and save them as a single file) or when I try to load them myself, the following error is produced:

> in main()  
> 104 # )  
> 105 # model = torch.nn.DataParallel(model).cuda()  
> → 106 model = vggpy.KitModel(“vggpy.npy”)  
> 107 model = torch.nn.DataParallel(model).cuda()  
> 108
> 
> ~\Documents\Pytorch\COCO Autoencoder\places365\vggpy.py in **init** (self, weight\_file)  
> 26 \_weights\_dict = load\_weights(weight\_file)  
> 27  
> —\> 28 self.conv1\_1 = self.\_\_conv(2, name=‘conv1\_1’, in\_channels=3, out\_channels=64, kernel\_size=(3, 3), stride=(1, 1), groups=1, bias=True)  
> 29 self.conv1\_2 = self.\_\_conv(2, name=‘conv1\_2’, in\_channels=64, out\_channels=64, kernel\_size=(3, 3), stride=(1, 1), groups=1, bias=True)  
> 30 self.conv2\_1 = self.\_\_conv(2, name=‘conv2\_1’, in\_channels=64, out\_channels=128, kernel\_size=(3, 3), stride=(1, 1), groups=1, bias=True)
> 
> ~\Documents\Pytorch\COCO Autoencoder\places365\vggpy.py in \__conv(dim, name, \*\*kwargs)  
> 124 layer.state\_dict()[‘weight’].copy_(torch.from\_numpy(\_weights\_dict[name][‘weights’]))  
> 125 if ‘bias’ in _weights\_dict[name]:  
> → 126 layer.state\_dict()[‘bias’].copy_(torch.from\_numpy(\_weights\_dict[name][‘bias’]))  
> 127 return layer  
> 128
> 
> RuntimeError: output with shape [64] doesn’t match the broadcast shape [1, 1, 1, 64]  
> ​

This is obviously happening on the first layer as it tries to load the bias terms, without anything being input to the model, but I’m not sure whether the problem is with the model shape or the _weights_, and how this might be modified to then work. The full code itself does appear accurate for [a constructed version of VGG](https://github.com/zhousteven/VGG16-PyTorch/blob/master/vgg.py):

```auto
import numpy as np
import torch
import torch.nn as nn
import torch.nn.functional as F
import math

_weights_dict = dict()

def load_weights(weight_file):
    if weight_file == None:
        return

    try:
        weights_dict = np.load(weight_file, allow_pickle=True).item()
    except:
        weights_dict = np.load(weight_file, allow_pickle=True, encoding='bytes').item()

    return weights_dict

class KitModel(nn.Module):

    
    def __init__ (self, weight_file):
        super(KitModel, self). __init__ ()
        global _weights_dict
        _weights_dict = load_weights(weight_file)

        self.conv1_1 = self.__conv(2, name='conv1_1', in_channels=3, out_channels=64, kernel_size=(3, 3), stride=(1, 1), groups=1, bias=True)
        self.conv1_2 = self.__conv(2, name='conv1_2', in_channels=64, out_channels=64, kernel_size=(3, 3), stride=(1, 1), groups=1, bias=True)
        self.conv2_1 = self.__conv(2, name='conv2_1', in_channels=64, out_channels=128, kernel_size=(3, 3), stride=(1, 1), groups=1, bias=True)
        self.conv2_2 = self.__conv(2, name='conv2_2', in_channels=128, out_channels=128, kernel_size=(3, 3), stride=(1, 1), groups=1, bias=True)
        self.conv3_1 = self.__conv(2, name='conv3_1', in_channels=128, out_channels=256, kernel_size=(3, 3), stride=(1, 1), groups=1, bias=True)
        self.conv3_2 = self.__conv(2, name='conv3_2', in_channels=256, out_channels=256, kernel_size=(3, 3), stride=(1, 1), groups=1, bias=True)
        self.conv3_3 = self.__conv(2, name='conv3_3', in_channels=256, out_channels=256, kernel_size=(3, 3), stride=(1, 1), groups=1, bias=True)
        self.conv4_1 = self.__conv(2, name='conv4_1', in_channels=256, out_channels=512, kernel_size=(3, 3), stride=(1, 1), groups=1, bias=True)
        self.conv4_2 = self.__conv(2, name='conv4_2', in_channels=512, out_channels=512, kernel_size=(3, 3), stride=(1, 1), groups=1, bias=True)
        self.conv4_3 = self.__conv(2, name='conv4_3', in_channels=512, out_channels=512, kernel_size=(3, 3), stride=(1, 1), groups=1, bias=True)
        self.conv5_1 = self.__conv(2, name='conv5_1', in_channels=512, out_channels=512, kernel_size=(3, 3), stride=(1, 1), groups=1, bias=True)
        self.conv5_2 = self.__conv(2, name='conv5_2', in_channels=512, out_channels=512, kernel_size=(3, 3), stride=(1, 1), groups=1, bias=True)
        self.conv5_3 = self.__conv(2, name='conv5_3', in_channels=512, out_channels=512, kernel_size=(3, 3), stride=(1, 1), groups=1, bias=True)
        self.fc6_1 = self.__dense(name = 'fc6_1', in_features = 25088, out_features = 4096, bias = True)
        self.fc7_1 = self.__dense(name = 'fc7_1', in_features = 4096, out_features = 4096, bias = True)
        self.fc8a_1 = self.__dense(name = 'fc8a_1', in_features = 4096, out_features = 365, bias = True)

    def forward(self, x):
        conv1_1_pad = F.pad(x, (1, 1, 1, 1))
        conv1_1 = self.conv1_1(conv1_1_pad)
        relu1_1 = F.relu(conv1_1)
        conv1_2_pad = F.pad(relu1_1, (1, 1, 1, 1))
        conv1_2 = self.conv1_2(conv1_2_pad)
        relu1_2 = F.relu(conv1_2)
        pool1_pad = F.pad(relu1_2, (0, 1, 0, 1), value=float('-inf'))
        pool1, pool1_idx = F.max_pool2d(pool1_pad, kernel_size=(2, 2), stride=(2, 2), padding=0, ceil_mode=False, return_indices=True)
        conv2_1_pad = F.pad(pool1, (1, 1, 1, 1))
        conv2_1 = self.conv2_1(conv2_1_pad)
        relu2_1 = F.relu(conv2_1)
        conv2_2_pad = F.pad(relu2_1, (1, 1, 1, 1))
        conv2_2 = self.conv2_2(conv2_2_pad)
        relu2_2 = F.relu(conv2_2)
        pool2_pad = F.pad(relu2_2, (0, 1, 0, 1), value=float('-inf'))
        pool2, pool2_idx = F.max_pool2d(pool2_pad, kernel_size=(2, 2), stride=(2, 2), padding=0, ceil_mode=False, return_indices=True)
        conv3_1_pad = F.pad(pool2, (1, 1, 1, 1))
        conv3_1 = self.conv3_1(conv3_1_pad)
        relu3_1 = F.relu(conv3_1)
        conv3_2_pad = F.pad(relu3_1, (1, 1, 1, 1))
        conv3_2 = self.conv3_2(conv3_2_pad)
        relu3_2 = F.relu(conv3_2)
        conv3_3_pad = F.pad(relu3_2, (1, 1, 1, 1))
        conv3_3 = self.conv3_3(conv3_3_pad)
        relu3_3 = F.relu(conv3_3)
        pool3_pad = F.pad(relu3_3, (0, 1, 0, 1), value=float('-inf'))
        pool3, pool3_idx = F.max_pool2d(pool3_pad, kernel_size=(2, 2), stride=(2, 2), padding=0, ceil_mode=False, return_indices=True)
        conv4_1_pad = F.pad(pool3, (1, 1, 1, 1))
        conv4_1 = self.conv4_1(conv4_1_pad)
        relu4_1 = F.relu(conv4_1)
        conv4_2_pad = F.pad(relu4_1, (1, 1, 1, 1))
        conv4_2 = self.conv4_2(conv4_2_pad)
        relu4_2 = F.relu(conv4_2)
        conv4_3_pad = F.pad(relu4_2, (1, 1, 1, 1))
        conv4_3 = self.conv4_3(conv4_3_pad)
        relu4_3 = F.relu(conv4_3)
        pool4_pad = F.pad(relu4_3, (0, 1, 0, 1), value=float('-inf'))
        pool4, pool4_idx = F.max_pool2d(pool4_pad, kernel_size=(2, 2), stride=(2, 2), padding=0, ceil_mode=False, return_indices=True)
        conv5_1_pad = F.pad(pool4, (1, 1, 1, 1))
        conv5_1 = self.conv5_1(conv5_1_pad)
        relu5_1 = F.relu(conv5_1)
        conv5_2_pad = F.pad(relu5_1, (1, 1, 1, 1))
        conv5_2 = self.conv5_2(conv5_2_pad)
        relu5_2 = F.relu(conv5_2)
        conv5_3_pad = F.pad(relu5_2, (1, 1, 1, 1))
        conv5_3 = self.conv5_3(conv5_3_pad)
        relu5_3 = F.relu(conv5_3)
        pool5_pad = F.pad(relu5_3, (0, 1, 0, 1), value=float('-inf'))
        pool5, pool5_idx = F.max_pool2d(pool5_pad, kernel_size=(2, 2), stride=(2, 2), padding=0, ceil_mode=False, return_indices=True)
        fc6_0 = pool5.view(pool5.size(0), -1)
        fc6_1 = self.fc6_1(fc6_0)
        relu6 = F.relu(fc6_1)
        drop6 = F.dropout(input = relu6, p = 0.5, training = self.training, inplace = True)
        fc7_0 = drop6.view(drop6.size(0), -1)
        fc7_1 = self.fc7_1(fc7_0)
        relu7 = F.relu(fc7_1)
        drop7 = F.dropout(input = relu7, p = 0.5, training = self.training, inplace = True)
        fc8a_0 = drop7.view(drop7.size(0), -1)
        fc8a_1 = self.fc8a_1(fc8a_0)
        prob = F.softmax(fc8a_1)
        return prob

    @staticmethod
    def __dense(name, **kwargs):
        layer = nn.Linear(**kwargs)
        layer.state_dict()['weight'].copy_(torch.from_numpy(_weights_dict[name]['weights']))
        if 'bias' in _weights_dict[name]:
            layer.state_dict()['bias'].copy_(torch.from_numpy(_weights_dict[name]['bias']))
        return layer

    @staticmethod
    def __conv(dim, name, **kwargs):
        if dim == 1: layer = nn.Conv1d(**kwargs)
        elif dim == 2: layer = nn.Conv2d(**kwargs)
        elif dim == 3: layer = nn.Conv3d(**kwargs)
        else: raise NotImplementedError()

        layer.state_dict()['weight'].copy_(torch.from_numpy(_weights_dict[name]['weights']))
        if 'bias' in _weights_dict[name]:
            layer.state_dict()['bias'].copy_(torch.from_numpy(_weights_dict[name]['bias']))
        return layer

```

Trying `_weights_dict[name][‘bias’].flatten()` and `_weights_dict[name][‘bias’].reshape(1,1,1,64)` didn’t help anything, either.

---

<div class="post-metadata">

**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\
**Post date:** [April 21, 2021, 6:21am UTC](https://discuss.pytorch.org/t/runtimeerror-output-shape-doesnt-match-broadcast-shape-on-weight-loading/118809/2 "2021-04-21T06:21:16Z")

</div>

Removing the additional dimensions should work as shown in this code snippet:

```python
conv = nn.Conv2d(3, 64, 3, 1, 1)
bias = conv.bias.clone()

conv.state_dict()['bias'].copy_(bias) # works
conv.state_dict()['bias'].copy_(bias[None, None, None, :])
> RuntimeError: output with shape [64] doesn't match the broadcast shape [1, 1, 1, 64]

```

The same error message is raised, if I add these additional dimensions to the `bias`, while it works for the matching shape.

Where did you try to remove the dimensions in your code?

---

<div class="post-metadata">

**Author:** ![cpeters](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/c/e274bd/32.png) [@cpeters](https://discuss.pytorch.org/u/cpeters)\
**Post date:** [April 21, 2021, 8:37am UTC](https://discuss.pytorch.org/t/runtimeerror-output-shape-doesnt-match-broadcast-shape-on-weight-loading/118809/3 "2021-04-21T08:37:43Z")

</div>

> [@ptrblck](#):
>
> `[None, None, None, :]`

I was trying to remove dimensions in the loading, but I wasn’t quite sure what had the wrong dimensions and I entirely forgot about `np.squeeze()`, so I was trying to use `flatten`, which didn’t help. `layer.state_dict()['bias'].copy_(np.squeeze(torch.from_numpy(_weights_dict[name]['bias'])))` works and loads correctly.

---

<div class="post-metadata">

**Author:** ![yathish](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/yathish/32/38308_2.png) [@yathish](https://discuss.pytorch.org/u/yathish)\
**Post date:** [May 15, 2021, 2:23am UTC](https://discuss.pytorch.org/t/runtimeerror-output-shape-doesnt-match-broadcast-shape-on-weight-loading/118809/4 "2021-05-15T02:23:23Z")

</div>

thanks much! was debugging another pre-trained caffe model, this did the trick (np.squeeze)