In deep learning code by pytorch

1 
# In[55]:
class ResBlock(nn.Module):
def __init__(self, n_chans):
super(ResBlock, self).__init__()
self.conv = nn.Conv2d(n_chans, n_chans, kernel_size=3,
padding=1, bias=False)
self.batch_norm = nn.BatchNorm2d(num_features=n_chans)
torch.nn.init.kaiming_normal_(self.conv.weight,
nonlinearity='relu')
torch.nn.init.constant_(self.batch_norm.weight, 0.5)
torch.nn.init.zeros_(self.batch_norm.bias)
def forward(self, x):
out = self.conv(x)
out = self.batch_norm(out)
out = torch.relu(out)
return out + x

can anyone tell me waht is code doing by steps ,please?
What are the sources that you can follow to understand the codes with myself,please ?

2

PyTorch docs provide the mathematical equations taking place behind the scenes for every operation where it is relevant.

For instance, nn.Conv2d:

IMG_20231208_113310
IMG_20231208_1133102431×1089 209 KB

For nn.Batchnorm2d:

IMG_20231208_113551
IMG_20231208_1135512453×935 244 KB

The other 3 lines in the init are filling the weights and biases with certain values before training begins.

The forward pass is then taking any tensors sent into the model and passing them through those two layers. But the torch.relu(out) will probably give an error because I don’t believe that exists. Should be F.relu(out) or torch.nn.functional.relu(out).

3

If you really want to understand what’s going on, you have to ignore PyTorch code and get familiar with the basic, in this case:

  • Convolutional Layers
  • Batch Normalization (vs. Layer Normalization)
  • Activation functions (here: ReLU)
  • Initialization of weights (here: Kaiming initialization)
  • Residual layers (core idea behind ResNet)

The whole purpose of PyTorch, Tensorflow, etc. is essentially to abstract from these nitty-gritty details. Of course, it still make sense to have at least some understanding, simply for understanding the input parameters of the different layers.