Hi all,
I am working with reimplementing a model in PyTorch and reusing pretrained weights. I have encountered errors which I thought are coming from floating point related problems.
I am providing a simple example which shows difference between convolution over input of ones vs summation of filter weights.
import torch
from torch.nn import functional as F
torch.backends.cudnn.deterministic = True
torch.backends.cudnn.benchmark = False
torch.manual_seed(0)
inputs = torch.ones((1, 2, 2, 2))
filters = torch.randn((1, 2, 2, 2))
res1 = F.conv2d(inputs, filters).numpy()[0, 0, 0, 0]
res2 = filters.sum().numpy()
print(res1, res2, res1 - res2)
# -1.6045358 -1.6045356 -2.3841858e-07
Is it something expected or is the error too big for such a small operation?