I have a question about using the “inplace” option when composing pytorch layers.
Consider the following LinBlock class:
class LinBlock(nn.Module):
def __init__(self,in_size,out_size,p_drop):
drop = nn.Dropout(p_drop,inplace=True)
lin = nn.Linear(in_size,out_size)
activ = nn.ReLU(inplace=True)
bn = nn.BatchNorm1d(out_size)
def forward(self,x):
self.drop(x)
x = self.lin(x)
self.activ(x)
x = self.bn(x)
return(x)
def forward2(self,x):
return self.bn(self.activ(self.lin(self.drop(x))))
Will the forward and forward2 functions do the same thing, as I believe they should? And is one of them for some reason faster?