Should I remove Dropout layer when testing my trained model?

1

Hi Everyone.
I Have trained a convolutional neural network with below structure.

### Defining Network
class Net(nn.Module):
    def __init__(self):
        super(Net,self).__init__()
        self.conv1 = nn.Conv2d(3,64,kernel_size=7, stride=1, padding=2)
        self.drop1 = nn.Dropout2d(p=0.1)
        self.maxpool1 = nn.MaxPool2d(5,stride=2)
        self.relu1 = nn.ReLU()
        self.conv2 = nn.Conv2d(64,128,kernel_size=5, stride=1, padding=2)
        self.drop2 = nn.Dropout2d(p=0.2)
        ......
        self.fc1 = nn.Linear(3*3*64, 64)
        self.drop4 = nn.Dropout(p=0.5)
        self.fc2 = nn.Linear(64,2)
        self.drop5 = nn.Dropout(p=0.5)
        
    def forward(self,x):
        x = self.conv1(x)
        x = self.drop1(x)
        x = self.maxpool1(x)
        x = self.relu1(x)
        x = self.conv2(x)
        x = self.drop2(x)
       .....
        x = self.fc1(x)
        x = self.drop4(x)
        x = self.fc2(x)
        x = self.drop5(x)
        return x

In this model, also, I used dropout layer to prevent overfitting problem. Actually, when testing the trained model, should I remove the Dropout layer?

2 Likes
2

You do not need to remove the Dropout layers in testing but you need to call model.eval() before testing. Calling this will change the behavior of layers such as Dropout, BatchNorm, etc. so that Dropout layers, for example, will not affect the result.

11 Likes
3

Thanks a lot for your help dear friend.

1 Like
4

Does removing the dropout layer have the same result with using model.eval()?
Thanks!

5

Removing layers from network just for the inference may introduce you to additional errors.
It is much simpler to do model.eval()