torch.optim.Adam for example has internal states, as it computes the running averages of gradient and its square.
If you don’t need to reset the optimizer (there might be use cases I’m not aware of), I would recommend to initialize it once and just use it inside the training loop.