I am working with reimplementing a model in PyTorch and reusing pretrained weights. I have encountered errors which I thought are coming from floating point related problems.
I am providing a simple example which shows difference between convolution over input of ones vs summation of filter weights.
import torch from torch.nn import functional as F torch.backends.cudnn.deterministic = True torch.backends.cudnn.benchmark = False torch.manual_seed(0) inputs = torch.ones((1, 2, 2, 2)) filters = torch.randn((1, 2, 2, 2)) res1 = F.conv2d(inputs, filters).numpy()[0, 0, 0, 0] res2 = filters.sum().numpy() print(res1, res2, res1 - res2) # -1.6045358 -1.6045356 -2.3841858e-07
Is it something expected or is the error too big for such a small operation?