RuntimeError: The expanded size of the tensor (64) must match the existing size (66) at non-singleton dimension 2. Target sizes: [256, 66, 64]. Tensor sizes: [256, 66]

vision
1

When I run the following (unchanged) code in stable PyTorch 1.9, I get the following error. How can I diagnose and fix it?

(fashcomp) [jalal@goku fashion-compatibility]$ python main.py --test --l2_embed --resume runs/nondisjoint_l2norm/model_best.pth.tar --datadir ../../../data/fashion/
/scratch3/venv/fashcomp/lib/python3.8/site-packages/torchvision/transforms/transforms.py:310: UserWarning: The use of the transforms.Scale transform is deprecated, please use transforms.Resize instead.
  warnings.warn("The use of the transforms.Scale transform is deprecated, " +
=> loading checkpoint 'runs/nondisjoint_l2norm/model_best.pth.tar'
=> loaded checkpoint 'runs/nondisjoint_l2norm/model_best.pth.tar' (epoch 5)
/scratch3/venv/fashcomp/lib/python3.8/site-packages/torch/nn/functional.py:718: UserWarning: Named tensors and all their associated APIs are an experimental feature and subject to change. Please do not use them for anything important until they are released as stable. (Triggered internally at  /pytorch/c10/core/TensorImpl.h:1156.)
  return torch.max_pool2d(input, kernel_size, stride, padding, dilation, ceil_mode)
Traceback (most recent call last):
  File "main.py", line 312, in <module>
    main()    
  File "main.py", line 149, in main
    test_acc = test(test_loader, tnet)
  File "main.py", line 244, in test
    embeddings.append(tnet.embeddingnet(images).data)
  File "/scratch3/venv/fashcomp/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
    return forward_call(*input, **kwargs)
  File "/scratch3/research/code/fashion/fashion-compatibility/type_specific_network.py", line 119, in forward
    masked_embedding = masked_embedding / norm.expand_as(masked_embedding)
RuntimeError: The expanded size of the tensor (64) must match the existing size (66) at non-singleton dimension 2.  Target sizes: [256, 66, 64].  Tensor sizes: [256, 66]

Related issue: https://github.com/mvasil/fashion-compatibility/pull/13Regarding RuntimeError adding keepdim by qa276390 · Pull Request #13 · mvasil/fashion-compatibility · GitHub

2

The error is raised, as norm cannot be expanded to masked_embedding:

masked_embedding = torch.randn(256, 66, 64)
norm = torch.randn(256, 66)
norm.expand_as(masked_embedding)
> RuntimeError: The expanded size of the tensor (64) must match the existing size (66) at non-singleton dimension 2.  Target sizes: [256, 66, 64].  Tensor sizes: [256, 66]

so you might want to add the missing dimension:

masked_embedding = torch.randn(256, 66, 64)
norm = torch.randn(256, 66).unsqueeze(2)
norm.expand_as(masked_embedding)
1 Like
3

changed line 118 of type_specific_network.py

from:
norm = torch.norm(masked_embedding, p=2, dim=2) + 1e-10
to:
norm = torch.norm(masked_embedding, p=2, dim=2, keepdim=True) + 1e-10