Expected scalar type Long but found Float

Hello! I have problem with my code:

import numpy as np
import os
import torch
from torch.utils.data import Dataset, DataLoader
import torchvision.models as models
import torchvision as trv
import torch.nn as nn
from sklearn.model_selection import train_test_split
import torch.optim as optim
import argparse


class CustomData(Dataset):
   def __init__(self, path, transform=None):
      try:
         self.path = path
         self.data = np.load(path).astype('float32')
         self.data = self.data.astype(np.int_)
         self.transform = transform
      except:
         print("error")


   def __len__(self):
      return len(self.data)

   def __getitem__(self, idx):
      if torch.is_tensor(idx):
         idx = idx.tolist()

      features = torch.tensor(self.data[idx, 2:])
      label = self.data[idx, 1]
      if self.transform:
         features = self.transform(features)
      return features, label





class CNN(nn.Module):
   def __init__(self, input_size, num_classes):
      super(CNN, self).__init__()
      self.conv1 = nn.Conv1d(1, 10, 1)
      self.conv2 = nn.Conv2d(1, 20, 45)
      self.fc1 = nn.Linear(45 * 20, 50)
      self.fc2 = nn.Linear(50, num_classes)
      self.bn1 = nn.BatchNorm1d(10)
      self.bn2 = nn.BatchNorm1d(20)
      self.bn3 = nn.BatchNorm1d(50)
      self.activation = nn.Sigmoid()

   def forward(self, x):
      x = x.unsqueeze(1)
      x = x.unsqueeze(1)
      x = self.activation(self.bn1(self.conv1(x)))
      x = self.activation(self.bn2(self.conv2(x)))
      x = x.view(-1, x.shape[1] * x.shape[2])
      x = self.activation(self.bn3(self.fc1(x)))
      x = self.fc2(x)

      return x


device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")

parser = argparse.ArgumentParser(description='PyTorch Feature Extraction')
parser.add_argument('--path', metavar='DIR',
                    help='path to dataset')
parser.add_argument('--j', '--workers', default=4, type=int, metavar='N',
                    help='number of data loading workers (default: 4)')
parser.add_argument('--b', '--batch-size', default=10, type=int,
                    metavar='N', help='mini-batch size (default: 256)')

args = parser.parse_args()

path = "final_relebeled_dataset.npy"
dataset = CustomData(path, transform=None)
dataloader = DataLoader(dataset, batch_size=args.b, shuffle=True, num_workers=args.j, drop_last=True)


# num_classes = 41
num_classes = 24
input_size = 53

# model = Simple_FC(input_size = input_size, num_classes = num_classes).to(device)
model = CNN(input_size=input_size, num_classes=num_classes).to(device)

lr = 0.01
momentum = 0.9
criterion = nn.CrossEntropyLoss()
optimizer = optim.SGD(model.parameters(), lr=lr, momentum=momentum)

def train(epoch):
   model.train()
   for (features, labels) in enumerate(dataloader):
      features = torch.tensor([features])
      labels = labels[0]
      print(type(features))
      features = features.type(torch.LongTensor)
      print(type(features))
      features, labels = features.to(device), labels.to(device)
      optimizer.zero_grad()
      print(features)
      output = model(features)
      loss = criterion(output, labels)
      loss.backward()
      optimizer.step()
      print(loss)


if __name__ == '__main__':
   for epoch in range(0, 10):
      train(epoch)

After I have started my code I see error :" RuntimeError: expected scalar type Long but found Float ". Could you help me solve my problem ?? Thank you !!!

Most likely this error is raised in:

loss = criterion(output, labels)

if you try to pass labels as a FloatTensor, while a LongTensor is expected.
Try to use:

loss = criterion(output, labels.long())

instead.

Thanks for your answer , but I have started study Traceback and I have seen that problem is in :

output = model(features)

I have tried to use:

output = model(features.long())

but I have the same error again.

Can you post the full traceback if the above fix didn’t work? features.long() seems unusual especially for a CNN model .

It is Traceback:
Traceback (most recent call last):
File β€œβ€, line 124, in
train(epoch)
File β€œβ€, line 105, in train
output = model(features.long())
File β€œβ€, line 889, in _call_impl
result = self.forward(*input, **kwargs)
File β€œβ€, line 56, in forward
x = self.activation(self.bn1(self.conv1(x)))
File β€œβ€, line 889, in _call_impl
result = self.forward(*input, **kwargs)
File β€œβ€, line 263, in forward
return self._conv_forward(input, self.weight, self.bias)
File β€œβ€, line 259, in _conv_forward
return F.conv1d(input, weight, bias, self.stride,
RuntimeError: expected scalar type Long but found Float

What is the purpose of

         self.data = np.load(path).astype('float32')
         self.data = self.data.astype(np.int_)

in the Dataset?
Can you try removing the second line?
You should also change the model invocation back to output = model(features).

The error message is confusing because the operator expected weights to be Long for Long input. To fix this you want to change your input to Float and match the weights which are already Float by default.

Thank for you answer but I tried it (what you have written) and it is the same error.

Can you double check the type of x in the model before the conv layer? e.g., print(x.dtype) should give float.

It has given torch.int64. Should it be float ??

Yes :wink: it should be float here

Thank you ! I have float now. I have another question. Now I have error :

Traceback (most recent call last):
  File "", line 121, in <module>
    train(epoch)
  File "", line 102, in train
    output = model(features)
  File "", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "y", line 56, in forward
    x = self.activation(self.bn1(self.conv1(x)))
  File "", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "", line 135, in forward
    return F.batch_norm(
  File "", line 2147, in batch_norm
    _verify_batch_size(input.size())
  File "", line 2114, in _verify_batch_size
    raise ValueError("Expected more than 1 value per channel when training, got input size {}".format(size))
ValueError: Expected more than 1 value per channel when training, got input size torch.Size([1, 10, 1])

What does it mean ?? Should I change sth in :

self.conv1 = nn.Conv1d(1, 10, 1)

Something seems strange about the output shape after a conv layer then. Can you separate the

x = self.activation(self.bn1(self.conv1(x)))

to

x = self.conv1(x)
x = self.bn1(x)
...

and print the shape of x in between to debug? It looks like the shape of x is smaller than expected for the batchnorm.

I have done it and problem is in :

x = self.bn1(x)

So should I change value in bn1 in init??

If you know what the expected shape and what change should be made to bn1, then sure.

I know that x.shape is torch.Size([1, 10, 1]), so value in BatchNorm1d should be
10, but I still have the same error :frowning:

The batchnorm layer would need more than a single value to calculate the stats (mean and var) from the input batch, so you would either need to use more than a single sample (increase the batch size) during training or increase the temporal dimension:

bn = nn.BatchNorm1d(10)
x = torch.randn(1, 10, 1)
out = bn(x)
> ValueError: Expected more than 1 value per channel when training, got input size torch.Size([1, 10, 1])

x = torch.randn(2, 10, 1)
out = bn(x) # works

x = torch.randn(1, 10, 2)
out = bn(x) # works
1 Like

What if my input size is 1 and kernel size must be 1 too ? Can I do sth with this, without change my data ??

I’m unsure how the kernel size is related to the input shape of the batchnorm layer.
However, if the input contains only a single value for each channel (as is the case here), you won’t be able to use batchnorm layers in training mode, since they need to calculate the stats from the input.
Since it’s impossible to calculate the var from a single sample (it would create NaNs) and also subtracting the mean from a single value would create a zero output, you could remove these layers from the model.