I’m interested in SWA so I’m trying to use it, but I don’t know what to use. I don’t know the difference between the following blogs and docs swa.
which one is the lastest version of SWA torch.optim.utils_swa or torchcontrib.SWA and what is the difference?
docs : https://pytorch.org/docs/stable/optim.html
SWA was added into PyTorch
1.6 in this PR, so it looks as if the contrib implementation was merged into the core.
EDIT: Here is also the blog post.