How to obtain model gradients with amp?

I occasionally like to inspect the gradients of a model but since I use amp I do not know anymore how to do that. Can you kindly help me how to obtain the scaled gradients?

I usually do:

            grads = []
            for p in model.parameters():
                if (p.grad is None) or (not p.requires_grad):

With amp, I do the following to train the model:

        assert parameters is not None
        self._scaler.unscale_(optimizer)  # unscale the gradients
        torch.nn.utils.clip_grad_norm_(parameters, clip_value, norm_type=norm_type)

Can somebody tell me how I can get the gradients of these model parameters in this case after I have scaled them? If I iterate again over parameters can I again get each gradient? Will it be scaled?

You can access the gradients in the same way as shown in your first approach after calling unscale manually as also shown in your second code snippet.

Thank you very much for your reply, @ptrblck , which was very helpful indeed to me.