I have two models that are identical and I’m trying to update one of them using the other in an EMA manner.
I’ve attempted to do this using the following code:
with torch.no_grad(): for teacher_param, student_param in zip(teacher.parameters(), student.parameters()): teacher_param.data *= tau teacher_param.data += (1 - tau) * student_param.data
Is this the correct approach to achieve my goal?
Thank you so much for your help!