How to make a tensor part of model parameters?

6

Hi Paulo. Good question. Am I sure that I sent this to the optimizer? Well, not explicitly, I just passed the entire model parameters as is conventional, and I assume that alpha was one of them, because after all, it was in the init method, so it should be part of the model. Can I pass it explicitly to the optimizer? How?

The other good question, how was alpha used in the code? I sent it as a paramter to a function that needed it, which is this:

def modrelu(re, im, alpha):
    abs_ = t.sqrt(re**2 + im**2)
    ang = t.atan2(im, re)
    abs_ = nn.functional.relu(abs_ + alpha)
    return abs_ * t.cos(ang), abs_ * t.sin(ang)

So, inside the forward of my neural net, at some point I call this function and I go like:

blah  =  modrelu( blah, blah, self.alpha )

Thanks.