I tried using the same approach that you mentioned for dynamically changing weight matrix.

But I am getting error as

RuntimeError: size mismatch, m1: [771 x 20], m2: [3 x 20] at /opt/conda/conda-bld/pytorch-cpu_1532576596369/work/aten/src/TH/generic/THTensorMath.cpp:2070

In the code I have to actually add atleast 12 hidden neurons dynamically but initially I tried using only one neuron. (trying cascade architecture with RPROP)

Code:-

class MultiLayerNet(torch.nn.Module):

def **init**(self, n_input, n_hidden, n_output):

super(MultiLayerNet, self).**init**()

self.sigmoid = torch.nn.Sigmoid()

self.relu = torch.nn.ReLU()

self.weight1 = nn.Parameter(torch.randn(n_input, 3)) # for all hidden layers

self.weight2 = nn.Parameter(torch.randn(2, 2)) # for output layer

```
def forward(self, x,count):
print(count)
print(x.shape, self.weight1)
if count == 1:
h_pred = self.relu(F.linear(x,self.weight1))
y_pred = self.sigmoid(F.linear(h_pred,self.weight2))
return y_pred
```

Note:-

input as x is of shape 771 x 20 and weight1 matrix is of size 20 x 3 which should actually get easily multiplied but I observed that in linear function instead of directly using the passed weight1 matrix it is using weight.t1() (transpose of the matrix) not sure why?

Further logs:-

in forward(self, x, count)

19 print(x.shape, self.weight1)

20 if count == 1:

—> 21 h_pred = self.relu(F.linear(x,self.weight1))

22

23 y_pred = self.sigmoid(F.linear(h_pred,self.weight2))

/usr/local/anaconda3/lib/python3.6/site-packages/torch/nn/functional.py in linear(input, weight, bias)

1024 return torch.addmm(bias, input, weight.t())

1025

-> 1026 output = input.matmul(weight.t())

1027 if bias is not None:

1028 output += bias