In case of more than 1 Neural network, which parameters are considered for backpropagation in pyTorch?

autograd
#1 
class Parent(nn.Module):

  def __init__(self,in_features,z_dim, img_dim):
        super().__init__()
        self.my_child1 = Child1 (z_dim, img_dim)
        self.my_child2 = Child2 (in_features)
      

  def forward(self,input):
         input=self.my_child1(input)
         input=self.my_child2(input)    
         return input
  
  def forward1(self,input):
         input=self.my_child1(input)
         return input
         
  def forward2(self,input):
         input=self.my_child2(input)
         return input


class Child2(nn.Module):
    def __init__(self, in_features):
        super().__init__()
        self.child2 = nn.Sequential(
            nn.Linear(in_features, 128),
            nn.LeakyReLU(0.01),
            nn.Linear(128, 1),
            nn.Sigmoid(),
        )

    def forward(self, x):
        return self.child2(x)


class Child1(nn.Module):
    def __init__(self, z_dim, img_dim):
        super().__init__()
        self.child1 = nn.Sequential(
            nn.Linear(z_dim, 256),
            nn.LeakyReLU(0.01),
            nn.Linear(256, img_dim),
            nn.Tanh(), 
        )

    def forward(self, x):
        return self.child1(x)

criterion=nn.BCELoss()
model=Parent(in_features,z_dim, img_dim)
output1=model.forward(noise)
loss=criterion(output1,torch.ones_like(output1))
loss.backward()

I have a parent module named as Parent and it has 2 child components packed in it. The child modules are defined below the parent component.

Now when loss.backward() is called, backpropagation is conducted with respect to which parameters? (Parameters of child1/child2 or both?)

What should I do if I need to conduct backpropagation on any of the child network? Could I take help of forward1() or forward2() method from the Parent module or do I need to call the child modules seperately?

#2

All parameters which were used in the forward pass will get the corresponding gradient accumulated into their .grad attribute.
In your example my_child1 and my_child2 are used and thus their parameters will get the gradients.

You could use your custom forwardX method, but note that you would break e.g. hooks.
PyTorch depends on the implementation of the forward method and will call it internally in its __call__ method. This is also the reason why you should call the model directly:

out = model(input)

instead of its forward method:

out = model.forward(input)
#3

Thanks a lot for the clarification! I have a requirement to build a Jacobian matrix where the row1 will have the partial derivative of the loss of the child1 with respect to all the parameters and the second row should have the loss of the child2 with repect to all the parameters. How can I build a matrix like that with the help of loss.backward() ?