.cuda() doesn't work for RuntimeError: Expected object of type torch.cuda.FloatTensor but found type torch.FloatTensor for argument #3 'weight'

1

Not sure why using .cuda() or .cuda(async=True) wouldn’t work here:

Here is what weight looks like before using .cuda()

weight: tensor([0.0526, 0.0081, 0.0043, 0.0250, 0.0108, 0.0192, 0.0073, 0.0385, 0.0175])

and below weight.is_cuda returns True

weight = weight.cuda(async=True)
print(weight.is_cuda)
model_ft = models.resnet50(pretrained=True)

num_ftrs = model_ft.fc.in_features
model_ft.fc = nn.Linear(num_ftrs, 9)

model_ft = model_ft.to(device)

criterion = nn.CrossEntropyLoss(weight=weight)

# Observe that all parameters are being optimized
optimizer_ft = optim.SGD(model_ft.parameters(), lr=0.001, momentum=0.9)
###optim.Adam(amsgrad=True)
# Decay LR by a factor of 0.1 every 7 epochs
exp_lr_scheduler = lr_scheduler.StepLR(optimizer_ft, step_size=7, gamma=0.1)
model_ft = train_model(model_ft, criterion, optimizer_ft, exp_lr_scheduler, num_epochs=100)

Error is:

RuntimeError                              Traceback (most recent call last)
<ipython-input-265-08b1f956820f> in <module>()
----> 1 model_ft = train_model(model_ft, criterion, optimizer_ft, exp_lr_scheduler, num_epochs=100)

<ipython-input-262-b7926afc6a1b> in train_model(model, criterion, optimizer, scheduler, num_epochs)
     48                     outputs = model(inputs)
     49                     _, preds = torch.max(outputs, 1)
---> 50                     loss = criterion(outputs, labels)
     51 
     52                     # backward + optimize only if in training phase

/scratch/sjn-p3/anaconda/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    475             result = self._slow_forward(*input, **kwargs)
    476         else:
--> 477             result = self.forward(*input, **kwargs)
    478         for hook in self._forward_hooks.values():
    479             hook_result = hook(self, input, result)

/scratch/sjn-p3/anaconda/anaconda3/lib/python3.6/site-packages/torch/nn/modules/loss.py in forward(self, input, target)
    860     def forward(self, input, target):
    861         return F.cross_entropy(input, target, weight=self.weight,
--> 862                                ignore_index=self.ignore_index, reduction=self.reduction)
    863 
    864 

/scratch/sjn-p3/anaconda/anaconda3/lib/python3.6/site-packages/torch/nn/functional.py in cross_entropy(input, target, weight, size_average, ignore_index, reduce, reduction)
   1548     if size_average is not None or reduce is not None:
   1549         reduction = _Reduction.legacy_get_string(size_average, reduce)
-> 1550     return nll_loss(log_softmax(input, 1), target, weight, None, ignore_index, None, reduction)
   1551 
   1552 

/scratch/sjn-p3/anaconda/anaconda3/lib/python3.6/site-packages/torch/nn/functional.py in nll_loss(input, target, weight, size_average, ignore_index, reduce, reduction)
   1405                          .format(input.size(0), target.size(0)))
   1406     if dim == 2:
-> 1407         return torch._C._nn.nll_loss(input, target, weight, _Reduction.get_enum(reduction), ignore_index)
   1408     elif dim == 4:
   1409         return torch._C._nn.nll_loss2d(input, target, weight, _Reduction.get_enum(reduction), ignore_index)

RuntimeError: Expected object of type torch.cuda.FloatTensor but found type torch.FloatTensor for argument #3 'weight'
2

Fixed it using

class_weights = torch.FloatTensor(weight).cuda()

from this reply;