What's the difference between nn.ReLU() and nn.ReLU(inplace=True)?

1

I implemented generative adversarial network using both nn.ReLU() and nn.ReLU(inplace=True). It seems that nn.ReLU(inplace=True) saved very small amount of memory.

What’s the purpose of the using inplace=True?
Is the behavior different in backpropagation?

34 Likes