Ah, sorry for the misunderstanding.
You can try below.
C = (A.unfold(2,80,1).unfold(3,80,1) - B.expand(b,c,21,21,80,80)).mean(dim=(-2,-1))
# variable version
_, _, H, W = A.shape
_, _, h, w = B.shape
C = (A.unfold(2,h,1).unfold(3,w,1) - B.expand(b,c,H-h+1,W-w+1,h,w)).mean(dim=(-2,-1))
The keys are torch.Tensor.unfold and torch.Tensor.expand.
The unfold function extracts sliding window and expand function views the tensor in expanded size without copying values.