Pytorch graph and multiple optimizers

1

Hi,

I have four feed forward networks, each designed using nn.Sequential() architecture and has its own optimizer and learning rate. I am using the output of three of them in one loss function and the output of the forth one in another loss function. My total loss is the sum of these two losses. When using .backward() on this total loss, are all of the parameters of these four networks being updated by four .steps()?

Thanks!

2

Yes.
Parameters of all four networks will be updated.

1 Like