Hi, this might be a simple question but when I would like to implement skip connections from ResNet or Densenet,
is it the number of channels or the dimension of the image that should be same for previous and current layer to do skip connection?
Thank you!
Hi @edshkim98
I am not 100% sure but the way I understand it (at least for ResNets), is that the skip connections handle different numbers of channels and the stride is handling the different dimensions of the image. If you look at a ResNet34 for example, after the first layer, you have 64 out_channels and in the second block of layer two you expect 128 in_channels. Since the skip connection skips the first block of layer two, you have to transform the 64 to 128 by e.g. applying 128 1x1x64 convolutions to match the number of channels. At the same time, since every new layer starts with a stride of 2, the dimensions of the images are halved at the beginning of every layer (note, I use the term layer as defined in the original paper: https://arxiv.org/abs/1512.03385).
If this is correct, I find the term ‘downsampling’ used in the implementation (https://pytorch.org/docs/stable/_modules/torchvision/models/resnet.html#resnet34) to be a bit confusing, since technically you ‘upsample’ the number of channels…
Hope this helps!
All the best
snowe