How to create a learnable input?

I want to create a learnable input matrix. So I create it as nn.Parameter. But I want to use a relu to make it non-negative and a softmax to make it normalized.

class Model(nn.Module):
    def __init__(self):
        super().__init__()
        self.weight = nn.Parameter(torch.randn(10, 10))
    
    def forward(self, x):
        self.weight = F.softmax(self.weight.relu(), dim=1)

The error is

cannot assign ‘torch.FloatTensor’ as parameter ‘weight’ (torch.nn.Parameter or None expected)

So what is the right way to do it? :pleading_face:

I am not sure of the context on what you are trying to achieve.
I guess, you do not need to assign self.weight again. It is already learnable and it will accumulate gradients when you backpropagate.

I want to create an adjacency matrix in GNN and want it to be learnable.
If I do not assign it in forward again, where should I apply relu and softmax

I see. One way is that, you can consider self.weight as logits and apply softmax in forward to get the adjacency matrix. (not sure why do we need relu), then continue with GNN operation.
But you do not need to assign it back to self.weight, as the logits will be learned when you loss.backward() and optimizer.step()

Oh, yeah. Now I find this question is stupid. I should assign it to another variable. Thanks for your reply :sweat_smile: