Using Pretrained resnet for regression, RuntimeError: Found dtype Double but expected Float

vision
1

Hello Everyone,
I am trying to train a model that learns to predict a single value(float) given an image using the pretrained resnet18 model. I am using the MSE loss but I am getting a Runtime error below.

RuntimeError: Found dtype Double but expected Float. from loss.backward()

Below is my code:


resnet18= models.resnet18(pretrained=True)
num_ftrs = resnet18.fc.in_features
resnet18.fc = nn.Linear(num_ftrs , 1)

def train_model(model, dataloaders, device, num_epochs=2, is_train=True):
  acc_history=[]
  loss_history=[]
  best_acc= 0.0
  for epoch in tqdm(range(num_epochs)):
    running_loss = 0.0
    running_corrects=0
  #Iterate over dataloader
    for x,y in dataloaders:

      for image,target in zip(x,y):
        image = image.to(device)
        target = target.to(device)
        model.to(device)

        #zero the parameter gradients 
        optimizer.zero_grad()
        criterion = nn.MSELoss()
        
        # forward 
        out = model(image.unsqueeze(0))
        print(out)
        loss = criterion(out, target)
        _,preds =torch.max(out,1)
        #backward 
        loss.backward()
        optimizer.step()

Your help is really appreciated

2

@ptrblck Kindly help out

3

Can you provide the full error? Also have you tried converting any of the variables to float use .float(). Maybe try doing any of these:

loss = criterion(out, target.float())
# or
loss = criterion(out.float(), target)
# or
loss = criterion(out, target).float()
4

Thanks for the response. I just tried all the above suggestions and below is the full error.

Epoch 0/1
----------
tensor([[0.9929]], grad_fn=<AddmmBackward>)
  0%|                                                                                                    | 0/2 [00:01<?, ?it/s]
Traceback (most recent call last):
  File "train.py", line 16, in <module>
    train_acc_hist, train_loss_hist = train_model(resnet18, train_loader, device)
  File "/model.py", line 64, in train_model
    loss.backward()
  File "/MTL/lib/python3.7/site-packages/torch/tensor.py", line 221, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph)
  File "/MTL/lib/python3.7/site-packages/torch/autograd/__init__.py", line 132, in backward
    allow_unreachable=True)  # allow_unreachable flag
RuntimeError: Found dtype Double but expected Float

I also printed out my input and target with their size and found this.

out : tensor([[-0.6273]], grad_fn=<AddmmBackward>) torch.Size([1, 1])
target : tensor(21.1924, dtype=torch.float64) torch.Size([])
loss : tensor(476.0975, dtype=torch.float64, grad_fn=<MseLossBackward>) torch.Size([])
5

@Dwight_Foster @ptrblck help out

6

dtypes of your target tensor as well as the image tensor should be torch.float32. Check your Dataset.__getitem__ returns the float32 tensors.

1 Like