Wrapping All Parameters of a Model with a Nonlinearity

1

I want to wrap all of my convolution parameters (.weight and .bias) with a sigmoid function which I want to make part of the graph i.e. sigmoid’s should also be part of the backprop. In other words, weight and bias values should never be out of range [0, 1].

What is the suggested way of doing this?

Thanks.

2

Try using torch.clamp over the model parameters.

3

Thanks, I am aware of torch.clamp. But, ignoring [0,1] range what if I want to use a specific function like sigmoid, square of the weights. Is there a way to this via hooks etc.?