I want to write a custom Linear/Quadratic regression function in Pytorch of the form-
def model(x):
pred = x @ W @ x.t() + x @ m + b
return pred
where M is an nxn matrix, m is an nx1 vector, and b is a scalar. The model function maps an n-dimensional vector to a scalar.
In the end, after training I also want to see the values of matrix W, vector m and scalar b.
I tried using the nn.Module to write my own version but wasn’t successful. Any help would be very appreciated in this direction.
Writing a custom model sounds like the right approach.
I assume you’ve taken a look at this or this tutorial? If so, could you describe where you are stuck right now?
First of all, thank you so much for the quick response.
I tried writing this custom module as the solution to my problem
class scr_model(torch.nn.Module):
def __init__(self):
"""
In the constructor we instantiate four parameters and assign them as
member parameters.
"""
super().__init__()
self.W = torch.nn.Parameter(torch.randn((26,26)))
self.m = torch.nn.Parameter(torch.randn((26,1)))
self.b = torch.nn.Parameter(torch.randn((1)))
def forward(self, x):
"""
In the forward function we accept a Tensor of input data and we must return
a Tensor of output data. We can use Modules defined in the constructor as
well as arbitrary operators on Tensors.
"""
return self.b + self.m * x + x *self.W * x.t()
But when i try to test it using
for x,y in train_loader:
preds = model(x)
print("Prediction is :\n",preds.shape)
print("\nActual targets is :\n",y.shape)
break
it shows me that
Prediction is :
torch.Size([26, 26])
In your code you are using the elementwise multiplication instead of the matmul operation from your previous post.
Change this and also the order of self.m and x and the output would have a shape of [1, 1]:
class scr_model(torch.nn.Module):
def __init__(self):
super().__init__()
self.W = torch.nn.Parameter(torch.randn((26,26)))
self.m = torch.nn.Parameter(torch.randn((26,1)))
self.b = torch.nn.Parameter(torch.randn((1)))
def forward(self, x):
return self.b + x @ self.m + x @ self.W @ x.t()
model = scr_model()
x = torch.randn(1, 26)
out = model(x)
print(out.shape)
# torch.Size([1, 1])
Note however that the output shape depends on the batch size which still sounds wrong.