Hello there!
I have two models that are identical and I’m trying to update one of them using the other in an EMA manner.
like this:
I’ve attempted to do this using the following code:
with torch.no_grad():
for teacher_param, student_param in zip(teacher.parameters(), student.parameters()):
teacher_param.data *= tau
teacher_param.data += (1 - tau) * student_param.data
Is this the correct approach to achieve my goal?
Thank you so much for your help!