Why would nn.ReLU cause my model to go over memory?

[Nikronic (Nikan Doosti)]

Hi,

Relu by default allocate new memory for output. You can modify input directly by setting inplace=True flag. Although, I am not sure this is the only reason or not but for sure Relu will consume memory.

What's the difference between nn.ReLU() and nn.ReLU(inplace=True)?

inplace=True means that it will modify the input directly, without allocating any additional output. It can sometimes slightly decrease the memory usage, but may not always be a valid operation (because the original input is destroyed). However, if you don’t see an error, it means that your use case is valid.

Bests