How to implement geometric sequence with a single nn.Parameter variable

1

I have the following definition in the init of my model
self.p = nn.Parameter(torch.ones(1))
My question is how to implement a geometric sequence based on self.p and use it during forward(), which is like
[p**i for i in range(5)], I have tried p_geometric = torch.tensor([self.p**i for i in range(x.size(1))]).cuda(), but the weight of self.p did not get updated with that implementation.
Thanks!

2

You shouldn’t recreate a tensor, as it would break the computation graph. Use torch.cat or torch.stack instead:

p = nn.Parameter(torch.ones(1) * 2)
out = torch.cat([p**i for i in range(10)])
out.mean().backward()
print(p.grad)
> tensor([409.7000])
3

Thanks! It works~ :grinning: