Apologies, I am new to quantization.
The current QAT tutorial on the official website runs the training entirely on CPU.
I realized the QAT model can be trained on GPU before calling torch.ao.quantization.convert on the model.
Will training the QAT model in GPU mode affect anything?
it will use cuda kernel instead of cpu kernel, this includes original model operators like batchnorm and the fake quantize operators that’s added by quantization flow, typically it won’t have much impact on accuracy.