Custom Linear Regression Layer

#1

I want to write a custom Linear/Quadratic regression function in Pytorch of the form-
def model(x):
pred = x @ W @ x.t() + x @ m + b
return pred
where M is an nxn matrix, m is an nx1 vector, and b is a scalar. The model function maps an n-dimensional vector to a scalar.

In the end, after training I also want to see the values of matrix W, vector m and scalar b.

I tried using the nn.Module to write my own version but wasn’t successful. Any help would be very appreciated in this direction.

#2

Writing a custom model sounds like the right approach.
I assume you’ve taken a look at this or this tutorial? If so, could you describe where you are stuck right now?

#3

First of all, thank you so much for the quick response.
I tried writing this custom module as the solution to my problem

class scr_model(torch.nn.Module):
    def __init__(self):
        """
        In the constructor we instantiate four parameters and assign them as
        member parameters.
        """
        super().__init__()
        self.W = torch.nn.Parameter(torch.randn((26,26)))
        self.m = torch.nn.Parameter(torch.randn((26,1)))
        self.b = torch.nn.Parameter(torch.randn((1)))

        

    def forward(self, x):
        """
        In the forward function we accept a Tensor of input data and we must return
        a Tensor of output data. We can use Modules defined in the constructor as
        well as arbitrary operators on Tensors.
        """
        return self.b + self.m * x +  x *self.W * x.t()

But when i try to test it using

for x,y in train_loader:
    preds = model(x)
    print("Prediction is :\n",preds.shape)
    print("\nActual targets is :\n",y.shape)
    break

it shows me that
Prediction is :
torch.Size([26, 26])

Actual targets is :
torch.Size([1, 1])

#4

In your code you are using the elementwise multiplication instead of the matmul operation from your previous post.
Change this and also the order of self.m and x and the output would have a shape of [1, 1]:

class scr_model(torch.nn.Module):
    def __init__(self):
        super().__init__()
        self.W = torch.nn.Parameter(torch.randn((26,26)))
        self.m = torch.nn.Parameter(torch.randn((26,1)))
        self.b = torch.nn.Parameter(torch.randn((1)))

    def forward(self, x):
        return self.b + x @ self.m  +  x @ self.W @ x.t()  
    
    
model = scr_model()
x = torch.randn(1, 26)
out = model(x)
print(out.shape)
# torch.Size([1, 1])

Note however that the output shape depends on the batch size which still sounds wrong.

1 Like
#5

Thank you so much Sir !!!