I have an autoencoder network with one shared encoder and two decoders for two different objects. For instance, I have the shared encoder and decoder_A and decoder_B. I am confused that when updating the weights, which one of the following scheme is correct:
(1) Run object A first: model(input_A, ‘A’), then compute loss_A, update encoder and decoder_A. Then run object B, model(input_B, ‘B’), compute loss_B, update encoder and decoder_B
(2) Run object A and B: model(input_A, ‘A’) model(input_B, ‘B’), compute loss = loss_A + loss_B, then update encoder, decoder_A and decoder_B.
Are these two methods completely equal? In method (2), will the gradients in the encoder will accumulate when model(input_A, ‘A’) and model(input_B, ‘B’) are processed sequentially.
Thanks for the help!