Resnet: batchnorm2D with in_place=False

[James_W]

thank you @ptrblck I have managed to solve this by setting broadcast_buffers=False It turns out that having it as True is an inplace operation.

but now, I have a massive memory leak issue which I have described here:

Massive memory leak when using distributed

I have a distributed code base I am trying to work with, but, with every epoch I see that my CPU memory increases almost linearly eventually running into OOM on a large 128GB machine :(( Without distributed, the code runs fine with no such issues. The isue is exactly describe here: CPU memory gradually leaks when num_workers > 0 in the DataLoader · Issue #13246 · pytorch/pytorch · GitHub I do use num_workers=16 but the solution posted there, using pyarrow, does not solve my issue - I still ha…

:((