I have implemented a convolutional autoencoder that perfectly works without weight sharing among encoder and decoder. I guess you all know how a conv. autoencoder works.
When tieing weights of the decoder to the encoder, i have noticed a weird behaviour of the weights of a standard nn.Conv2d:
For my case the input ist self.conv1 = nn.Conv2d(1,100,(16,5),stride=(16,5),padding=0), the auto-initialized weights for this layer are of size [100,1,16,5].
For the deconv I should use the the functional library with the transpose of these weights, right? This is the mathematically correct way to share weights. So what i would do looks like this
F.conv_transpose2d(out, weight=self.conv1.weight.transpose(0,1), bias=None, stride=(16,5),padding=0)
this throws an error, if I don’t transpose the weights in the conv_transpose2d it doesn’t throw an error.
So, this one works F.conv_transpose2d(out, weight=self.conv1.weight, bias=None,stride=(16,5),padding=0)
This seems like a weird behaviour (and maybe leads to errors in the future), especially because for fully connected (linear) layers it exactly works the way i would expect it to work.
Any ideas on this?
Thanks in advance