How to Initialize network output

1

I have a regression problem where a neural network predicts the output which have a range of 0 to inf. I know that the output should be close to some heuristic and would like to initialize the output accordingly.

In what way can I do this?

2

Change the weights of the last output layer to zero and bias to the initialisation value.

last_layer = nn.Linear(10, 10)
last_layer.weight.data.zero_()
last_layer.bias.data.fill_(1.0) # replace 1.0 with the required value

x = torch.randn(10)
last_layer(x)
# tensor([1., 1., 1., 1., 1., 1., 1., 1., 1., 1.], grad_fn=<AddBackward0>)

Essentially, “biasing” the output of the network. Hope this helps :slight_smile: