Adding autoencoders to large network causes memory issues

[ptrblck]

@Hrrsmjd

So adding a few Conv3d layers is somehow using at least 15GB of GPU memory.

This would be expected depending on the input activation size, as the intermediate forward activations could use the majority of the memory. You could check this post for an estimation of a ResNet architecture.