RuntimeError due to inplace operation in GAN generator architecture with skip connections

Since the out-of-place operation fixed the issue, I don’t think the linked post is related but might still be helpful to gain more insight into potential future issues.

The reason for the issue using inplace operations is described in this post with an example. As you can see, depending on the operation, the outputs are needed to calculate the gradients in the backward pass. If you manipulate them inplace, this won’t be possible anymore and the error is raised.

Also, I using inplace operations would also disallow the JIT to fuse these operations, so you might want to skip them entirely.

1 Like