When adding dropout layers to a Sequential module, does it matter whether I set the inplace parameter to True or False? Intuitively it seems like it would be safer to set inplace to True but is it fine if it is also False?
If you can use inplace or not depends on whether the layer that gives you the input for dropout needs is output (i.e. the dropout input) for the backward. This is not always obvious. The flip side is just that promise of inplace: It saves memory by reusing the memory for the activation.
In that sense the default inplace=False is the safer option. Indeed, I would venture that in almost all contexts inplace=True is a premature optimization.
Best regards
Thomas