Some fundemental concept problems wanted to figure out

Hello, I am wondering how do the part code do in a training loop that I saw them frequently used.
Especially update and scale do.


Also, in what situation we need to use torch.amp.autocast() ?

Thank you

Automatic Mixed Precision training (via torch.cuda.amp) can be used to speed up your training and I think the best place to get started is to check out the docs, the Automatic Mixed Precision recipe as well as the examples.

1 Like

Thank you very much!