Rename nn.Parameter

1

Hi all,

I have been trying to obtain the following behavior when defining an own model: In essence, what I would want is to have a Parameter whose name would be weights, so that in model.state_dict() I would see weights. But, for simplicity in coding, I would want to refer to it in the code as model.W, instead of model.weights. I have tried the following:

from torch.nn import Module, Parameter
from torch import randn

class mymodel(Module):
    def __init__(self):
        super(mymodel, self).__init__()
        W = Parameter(randn(5,4))
        self.register_parameter("weights", W)
        self.W = W

model = mymodel()

With this code, I can call model.W, but also model.weights. Plus, when I call model.state_dict(), then I obtain a list of two parameters, “W” and “weights”.

I wanted to ask, firstly, if the kind of behavior I would desire (being able to call only to model.W and having only “weights” in model.state_dict(), both referring to the same Parameter) is common/good practice, and if so, how to implement it properly.

Thanks in advance!

2

No! This is decidedly not good style. You are trading conceptual simplicity for saving a few characters (6 instead of 12).

Now, what you can do if your forward uses self.weights so often that you don’t want to spell it?
I think a perfectly good solution is to do a local assignment W = self.weights at the beginning of the forward (or whatever method uses W). If you use it often enough, it’ll even be less characters.

Best regards

Thomas

1 Like
3

I see. I was kind of expecting this would be the situation. I was looking for a simple way of being a bit more explicit on what W meant in a big code that I had already developed, but probably the simplest thing to do in terms of both explainability of the code and simplicity of the change is just renaming Wto weightseverywhere. Thank you very much for the clarification.

4

Actually I have the same demand, as I need to generate models in python and load my model in c++ and training in c++. I need to set parameter names dynamically in python and get all parameter names in c++, but if I do register, then I got different names point to same parameter.
How ever, I found out that python code will only set parameter name automatically if that parameter is direct class member. so my ugly solution is use a dict to cache parameters:

class NeuralNetwork(nn.Module):
def init(self):
super().init()
self.param_map = {}
v1 = torch.nn.Parameter(torch.tensor([[[1]]], dtype=torch.float32, requires_grad=True))
v2 = torch.nn.Parameter(torch.tensor([[[1]]], dtype=torch.float32, requires_grad=True))
self.register_parameter(‘weights1’, v1)
self.register_parameter(‘weights2’, v2)
self.param_map[‘v1’] = v1 # mimic self.v1
self.param_map[‘v2’] = v2