I’m using the functional interface to conv2d and am looking for an efficient way to define a weight matrix W which has the following constraint: W[i, j, a, b] = W[j, i, ks-1-a, ks-1-b] for all i, j, a, b. ks is the kernel size
How should I create and register W as a Parameter in my object constructor? It seems challenging to create given the tied weights within W.
I’ve tried defining the following in my constructor
W0 = nn.Parameter(torch.zeros(nf, nf, ks, ks), requires_grad=True)
and then creating a W in my forward function via
W0_flipped = flip(flip(W0, 3), 2) W = (W0 + W0_flipped.permute(1, 0, 2, 3)) / 2.0
where “flip” is defined as in https://github.com/pytorch/pytorch/issues/229. The resulting W has the appropriate constraints, but this definition comes at the cost of creating many redundant parameters in W0. I then use W in calls like
f = nn.functional.conv2d(inputs, W, bias=b, stride=1, padding=pad_f)
How can I create this without redundant parameters?