I want to implement a residual network, and I see that they work best if you start with an initial negative bias for the skip-connections (for example b = -1, -3, … ). My skip connections are 1x1 convolutions (since I need them for resizing) and I want to somehow initialize the biases of these layers with a negative value, for example:
self.skip_connection = nn.Conv2d(in_channels=3 , out_channels=16, kernel_size=1, stride=2, padding=0, BIAS_INITIALIZER= -3)
This does not work, since the
BIAS_INITIALIZER part is taken from tensorflow, but how can I do that here?