Some fundemental concept problems wanted to figure out

1

Hello, I am wondering how do the part code do in a training loop that I saw them frequently used.
Especially update and scale do.

        optimizer.zero_grad()
        scaler.scale(loss).backward()
        scaler.step(optimizer)
        scaler.update()

Also, in what situation we need to use torch.amp.autocast() ?

Thank you

2

Automatic Mixed Precision training (via torch.cuda.amp) can be used to speed up your training and I think the best place to get started is to check out the docs, the Automatic Mixed Precision recipe as well as the examples.

1 Like
3

Thank you very much!