Here I am sitting clueless that nn.Identity actually takes up a considerable amount of GPU memory when I though, in fact, this should be a no-op should it not?
Can someone elaborate on this?
it is a no-op, in a sense that only python bytecode of identity function is executed. your problem is likely elsewhere (note that cuda.memory_allocated() is not changed by nn.Identity, but you may be blocking tensor freeing somehow)