In Torch, I could use the following command:
cnn_params, cnn_grad_params = cnn:getParameters()
to get a 1D tensor of all the trainable parameters of a given model (and corresponding gradients).
Is it possible to do something like that in PyTorch so that cnn_params shares the same memory of the corresponding model? I should mention that I only care about the trainable parameters (i.e., weights and biases) and not their gradients.
Thanks a lot for any help!
You can get the parameters by calling
model.parameters(). This will return a generator, so if you need a list just call
Thanks for your answer ptrblck. My idea, if possible, would be to manipulate a 1D tensor with all the parameters so that I can avoid looping through the list and perform the same operation several times (e.g., replacing parameters above a certain value in each individual module) in order to speed up the algorithm.
torch.cat([param.view(-1) for param in model.parameters()])
Should work for you.
Thank you. I tried the solution that you suggested but it seems that this object does not share the same memory as the initial model.
Here is an example code:
net = nn.ModuleList()
net.append(nn.Conv2d(3, 8, 3))
net.append(nn.Conv2d(8, 8, 3))
net.append(nn.Conv2d(8, 1, 3))
unrolled = torch.cat([param.view(-1) for param in net.parameters()])
In case anyone else is interested, the closest solution I was able to find involves using
torch.nn.utils.parameters_to_vector() to get the parameters vector and then calling
torch.nn.utils.vector_to_parameters() when I’m done modifying it.
from torch.nn.utils import parameters_to_vector, vector_to_parameters
Also, this link may be a help: