By default parameter inPlace for nn.LeakyRelU has been set to False. I was wondering what are the repercussions for that, and why that is the default behavior?
I believe if we do not explicitly set it to True then a bit more memory would be used as a copy of that particular layer would need to be saved. LeakyRelu should not have any parameters (right?), so ideally IMHO inplace=True should be default behavior. Why would anyone want it to be False?