What's the proper order of "model.to(device)" and "optim(model.parameters())", thanks

what’s the proper order of “model.to(device)” and “optim(model.parameters())”, thanks

  1. model.to('cuda')
  2. opt(model.parameters(), ...)

From the docs:

If you need to move a model to GPU via .cuda(), please do so before constructing optimizers for it. Parameters of a model after .cuda() will be different objects with those before the call.

1 Like

Thanks! I also want to know what the consequences will be if I don’t do so?

In the worst case the optimizer might not update the passed parameters, so I would recommend to stick to the docs.