I’m removing filters from a convolutional neural network. So I modified the weight and bias matrices of all layers accordingly. As including ‘momentum’ in SGD optimizer will generate a momentum values matrix equivalent to each module’s (i.e Conv weight, Conv bias, BatchNorm weight etc.) shape. I modified the momentum matrix shape too. But it threw the following error
Traceback (most recent call last):
File “/content/drive/MyDrive/lenet_prune/custom_loss2.py”, line 482, in
optimizer.step()
File “/usr/local/lib/python3.7/dist-packages/torch/optim/lr_scheduler.py”, line 65, in wrapper
return wrapped(*args, **kwargs)
File “/usr/local/lib/python3.7/dist-packages/torch/optim/optimizer.py”, line 89, in wrapper
return func(*args, **kwargs)
File “/usr/local/lib/python3.7/dist-packages/torch/autograd/grad_mode.py”, line 27, in decorate_context
return func(*args, **kwargs)
File “/usr/local/lib/python3.7/dist-packages/torch/optim/sgd.py”, line 117, in step
nesterov)
File "/usr/local/lib/python3.7/dist-packages/torch/optim/functional.py", line 169, in sgd
buf.mul(momentum).add_(d_p, alpha=1 - dampening)
RuntimeError: The size of tensor a (19) must match the size of tensor b (20) at non-singleton dimension 0