In the resnet50 model from torchvision, the bottlneck layer is given as below.

Why isnt there a nonlinearity (ReLU) after each Conv2d (or BatchNorm2d)?

I see only one nonlinearity after all the 3 Convolutions.

Thanks

(layer1): Sequential(

(0): Bottleneck(

(conv1): Conv2d (64, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)

(bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True)

(conv2): Conv2d (64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)

(bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True)

(conv3): Conv2d (64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)

(bn3): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True)

(relu): ReLU(inplace)

(downsample): Sequential(

(0): Conv2d (64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)

(1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True)

)

)

This is the model from torchvision.