Update: I have found something that almost works, namely:
for name, parameter in self.named_parameters():
newParam = myFunction(parameter)
setattr(self, name, newParam)
The issue now is that setattr does not work, as it complains that newParam is a Variable as opposed to a Parameter. I guess that even if we pass a Parameter to a function, the result is always a variable. How can I then set the parameter equal to the variable?
Doing something like nn.Parameter(newParam.data) clearly won’t work, because we won’t be able to backpropagate the gradients through myFunction. What do you suggest?