There is a tensor which I want to learn, except for one variable in the tensor. In fact, I would like the entry of this tensor not to be a variable and, moreover, change the value of this entry in the forward pass so that I can take into account two passes with two different values of that entry into my loss function. Is this possible and if so, how to achieve it?
Would concatenating the input with another tensor during the forward pass work?
torch.manual_seed(2809)
lin = nn.Linear(10, 1)
x = torch.randn(1, 9, requires_grad=True)
constant = torch.zeros(1, 1)
data = torch.cat((x, constant), dim=1)
out = lin(data)
out.mean().backward()
constant = torch.ones(1, 1)
data = torch.cat((x, constant), dim=1)
out = lin(data)
out.mean().backward()
print(x.grad)
print(lin.weight.grad)