Method inside forward() in nn.Module

Can we do backward() for some tensors inside forward()?
Can we change the device of certain sub modules or tensors inside forward()?

In my view, yes. You can do whatever you want if your solution to a task calls for such operations inside forward. forward is just an API like any other python API’s. As long as the autograd doesn’t throw errors, you can choose to code anything in forward.

Hi, InnovArul. I’m sorry to disturb you by using this way for that I really don’t have other ways for my problem. My problem is as shown in the link: Error in dist.scatter() when mpi backend is used
Could you please do me a favor?