Fuse_conv_bn_eval does not work as expected


My test code is like this:

import torch

conv = torch.nn.Conv2d(3, 32, 3, 1, 1)
bn = torch.nn.BatchNorm2d(32)

fused = torch.nn.utils.fuse_conv_bn_eval(conv, bn)

inten = torch.randn(2, 3, 224, 224)

out1 = bn(conv(inten))
out2 = fused(inten)
print((out1 - out2).abs().sum())

The output is tensor(0.1754, grad_fn=<SumBackward0>). Why is there so large difference between the separate computation and fused computation, and how could I make the fussd version work like separate version?

If you check the (out1 - out2).abs().max() value you’ll see that it’s in the range ~1e-6, which is most likely created due to the limited floating point precision.
Summing these small absolute errors might result in the larger difference.

Thanks for telling me that!!

I tried to replace one conv-bn pair with the fused conv in a trained model, and test the model on some test set. The acc drops almost 1%. It seems that the point precision error is not neglectful. How could I fuse conv-bn pair without losing accuracy please ?

If you think these changes are causing a drop in model performance, you could fine-tune the model with the fused layers for some iterations and check if this would recover the performance.

Thanks !!

By the way, is there any suggestions about finetune like this ? I plan to fine tune for 5 epochs, with the lr value and its annealing strategy not changed (I used cosine lr to train my model before fuse). Shall I use a smaller lr or have the lr annealing curve changed ?

Unfortunately I don’t know what the best approach would be.
If possible, I would try to add the fused layers from the beginning, but if your model is already trained and you would like to fuse these layers now, you would have to try out different approaches. :confused: