I have a backward hook function with ‘newLayer.register_backward_hook(hook_function)’ where I do not know how to control the inputs to it. The function contains a line like. def hook_function(self, grad_input, grad_output): self.average = self.average * 0.99 + grad_output[0].sum((0,2,3)) * 0.0…

I don’t know how self.average is initialized, but would assume this should work: self.average = self.average.to(grad_output[0].device) * 0.99 + grad_output[0].sum((0,2,3)) * 0.01 Could you check it and see, if you are still getting an error? In that case, could you post a small code snippet to re…

Multi GPU backwards hook on wrong device

pytorcher November 12, 2020, 5:02pm 12

Have another thread now that includes a full code sample.