RuntimeError: expanded size

1 
self.hidden_layer_1 = torch.nn.Linear(50,78)
self.output = torch.nn.Linear(78, 2)

def forward(self, input):
        H1 = self.hidden_layer_1(input)
        H1 = self.ReLU(H1)
        final_inputs = self.output(H1)
        # not applying activation on final_inputs because CrossEntropyLoss does that
        return final_inputs

input shape = (2425727, 50)

        weights = np.load('./w.npy')
        bias = np.load('./b.npy')
        print(weights.shape,bias.shape)# (50, 78) (78,)
        self.hidden_layer_1.weight = torch.nn.Parameter(torch.from_numpy(weights))
        self.hidden_layer_1.bias = torch.nn.Parameter(torch.from_numpy(bias))

Error - RuntimeError: The expanded size of the tensor (50) must match the existing size (78) at non-singleton dimension 1. Target sizes: [1024, 50]. Tensor sizes: [78]

2

Your weights and bias seem to have a wrong shape, since the default linear layer using your setup would use:

print(nn.Linear(50, 78).weight.shape)
> torch.Size([78, 50])
print(nn.Linear(50, 78).bias.shape)
> torch.Size([78])

so you migth need to fix it (permute the weights and make sure the bias has a single dimension).

1 Like