How do linear combination using pytorch?

1

I am building a network that creates filters in a linear combination. But I don’t know if the approach is right. And it is also slow.

And I want to use the return value as weight.
How do you do it?

class Layer (torch.nn.Module):
    def __init __ (self, in_ch, out_ch):
        super (Layer, self) .__ init __ ()
        self.out_ch = out_ch
        self.in_ch = in_ch
        self.weights = torch.nn.Parameter (torch.Tensor (out_ch, in_ch, 5))

    def forward (self, x):
        w = torch.empty (self.out_ch, self.in_ch, 3, 3) .to ('cuda')

        for i in range (self.out_ch):
            for j in range (self.in_ch):
                w [i] [j] = x [0] .mul (self.weights [i] [j] [0]) + x [1] .mul (self.weights [i] [j] [1]) + x [2] .mul (self.weights [i] [j] [2]) \
                            + x [3] .mul (self.weights [i] [j] [3]) + x [4] .mul (self.weights [i] [j] [4])
        return w

class Net (nn.Module):
    def __init__(self):
       self.layer = Layer (3, 16)
       
    def forward (self, x):
        weight = self.layer(x) # x shape is (3, 3, 3)
        x = F.conv2d (x, weight)
        return x
  1. Computation Speed ​​improvement
  2. How to update parameters with parameters
2

Hi,

You can use the version given in nn: torch.nn.Linear().

To update the parameters, you can use the builtin optimizers in gneeral. You should check the tutorial on how to write a training loop.

1 Like