I run net train and test in single GPU is good, and I try to train model in multi-gpus, and I found that the process of training is good, however, when I convert model into quantization, and the model acc is 1% in CIFAR100 which means random.
My convert code is as follows:
126 if gpus == 1:
127 quantized_model = torch.quantization.convert(net.eval().to(device), inplace=False)
128 else:
129 print("expert model from torch DataParallel")
130 single_net = deepcopy(net.module)
131 #single_net = net
132 quantized_model = torch.quantization.convert(single_net.eval().to(device), inplace=False)
133 #quantized_model = quantized_model.module
134 quantized_model.eval()
if gpu number is more than one, I use net = torch.nn.DataParallel(net), so when I need to convert it, I first do single_net = deepcopy(net.module), but the acc of quantized_model is 1% , well the training acc is about 77%. Meanwhile, it has warning:
/home/autolab/anaconda3/lib/python3.7/site-packages/torch/quantization/observer.py:208: UserWarning: Must run observer before calling calculate_qparams. Returning default scale and zero point.
Returning default scale and zero point.")
So if anyone has any ideas?