Quantization Aware Training (QAT) Choosing Hyperparameters

quantization
1

Hi there!

I am trying to use Quantization Aware Training to get better performance on my quantized EfficientNet model, since PTQ performed very poorly. I tried using the original hyperparameters from the normal model to train it, and it did not perform well. After decreasing the learning rate, I was able to get much higher performance, but still not very close to the original model.

Can anyone provide me with some guidance for hyperparameter tuning with QAT or any other tips and tricks that could be useful? I was not able to find any good resources with this kind of information, so if you could point me in the direction of those, that would also be helpful.

Thank you.

2

Hi @Vij_Flex, can you share what QConfigs you are using to quantize the model? Hyperparameter tuning is mostly orthogonal to quantization, so unfortunately there aren’t really any recommended ways to do that from the PyTorch architecture optimization side.

3

Hi @andrewor, I am using fbgemm for the QConfigs.

4

You can check Models and pre-trained weights — Torchvision 0.14 documentation for inspiration, there are a couple of recipes linked there for QAT (unfortunately not for EfficientNet but the hyperparameters from other vision models should be useful).