TypeError: empty() received an invalid combination of arguments - got (tuple, dtype=NoneType, device=NoneType), but expected one of: * (tuple of ints size, *, tuple of names names, torch.memory_format memory_format, torch.dtype dtype, torch.layout layout

nlp
1

I am trying to do a memes analysis using the fusion model. For this, I have extracted the text feature using a pre-trained BERT model and the image feature using EfficientNet_B7. After that, I am passing these two parameters to my custom model class.

class FusionNet(nn.Module):
    def __init__(self, num_classes, drop_prob = 0.1):
        super().__init__()
        
        self.concat_layer = nn.Linear(in_features=768+1000, out_features= 512)
        self.bn = nn.BatchNorm1d(512)
        self.bn1 = nn.BatchNorm1d(768)
        self.bn2 = nn.BatchNorm1d(1000)
        # self.dropout = nn.Dropout(drop_prob)
        self.linear = nn.Linear(512, num_classes)
        self.relu = nn.ReLU()
        self.softmax = nn.Softmax()
        
    def forward(self, image_features, text_features):
        text_features = self.bn1(text_features)
        image_features = self.bn2(image_features)
        fused_input =  torch.cat((text_features, image_features), dim=1)
        x = self.concat_layer(fused_input)
        x = self.relu(self.bn(x)) 
        x = self.dropout(x)
        x = self.softmax(self.classify(x)) 
        
        return x

While running the code, I am getting the following error:

Epoch 1/100
----------
  0%|          | 0/70000 [00:00<?, ?it/s]torch.Size([8, 1, 256, 256])
torch.Size([8, 1000]) torch.Size([8, 768])
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
File <timed exec>:10, in <module>

/home/guest/Code/memotion_2022/task_1.ipynb Cell 39' in train_epoch(model, data_loader, criterion, optimizer, device, n_examples)
     25 text_features = bert_output['pooler_output'].to(device)
     27 print(image_features.shape, text_features.shape)
---> 29 outputs = FusionNet(
     30     image_features,
     31     text_features
     32 )
     34 _, preds = torch.max(outputs, dim=1)
     35 loss = criterion(outputs, targets)

/home/guest/Code/memotion_2022/task_1.ipynb Cell 43' in FusionNet.__init__(self, num_classes, drop_prob)
      8 self.bn2 = nn.BatchNorm1d(1000)
      9 # self.dropout = nn.Dropout(drop_prob)
---> 10 self.linear = nn.Linear(512, num_classes)
     11 self.relu = nn.ReLU()
     12 self.softmax = nn.Softmax()

File /home/anaconda3/envs/koyel/lib/python3.8/site-packages/torch/nn/modules/linear.py:96, in Linear.__init__(self, in_features, out_features, bias, device, dtype)
     94 self.in_features = in_features
     95 self.out_features = out_features
---> 96 self.weight = Parameter(torch.empty((out_features, in_features), **factory_kwargs))
     97 if bias:
     98     self.bias = Parameter(torch.empty(out_features, **factory_kwargs))

TypeError: empty() received an invalid combination of arguments - got (tuple, dtype=NoneType, device=NoneType), but expected one of:
 * (tuple of ints size, *, tuple of names names, torch.memory_format memory_format, torch.dtype dtype, torch.layout layout, torch.device device, bool pin_memory, bool requires_grad)
 * (tuple of SymInts size, *, torch.memory_format memory_format, Tensor out, torch.dtype dtype, torch.layout layout, torch.device device, bool pin_memory, bool requires_grad)

I am not able to fix this error for the last two days.
Please help me out in identifying the issue :smile:
Thanks,
Neeraj

2

Make sure you are passing an int as the num_classes argument as it seems you are using a tuple as seen here:

# works    
model = FusionNet(num_classes=10)

# passing a tuple raises your error
model = FusionNet(num_classes=(10,))
# TypeError: empty() received an invalid combination of arguments - got (tuple, dtype=NoneType, device=NoneType), but expected one of:
#  * (tuple of ints size, *, tuple of names names, torch.memory_format memory_format, torch.dtype dtype, torch.layout layout, torch.device device, bool pin_memory, bool requires_grad)
#  * (tuple of SymInts size, *, torch.memory_format memory_format, Tensor out, torch.dtype dtype, torch.layout layout, torch.device device, bool pin_memory, bool requires_grad)
3

Thanks for your reply.
The issue has been resolved. I was using Class-name instead of the instance of that class, i.e. FusionNet(image_features, text_features) instead of Fusion_model(image_features, text_features).

4

in_features must be int, so use int(768+1000) instead

1 Like