Expected object of type torch.cuda.FloatTensor but found type torch.cuda.DoubleTensor for argument #4 'mat1'

Hi all,
I am developing a regression model and having some problems in data types’ alignments.
I read the data from the csv file and convert it to float64 using sklearn.preprocessing.StandardScaler() which I later convert to torch tensor. After that I save to into training.pt and test.pt format.

Problem:
Now the problem is that, it is giving error in the first fully connected layer of the network.
The error is the following:

Exception ignored in: <bound method _DataLoaderIter.__del__ of <torch.utils.data.dataloader._DataLoaderIter object at 0x7fefaf76cd30>>
Traceback (most recent call last):
  File "/home/hiwi/anaconda3/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 399, in __del__
    self._shutdown_workers()
  File "/home/hiwi/anaconda3/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 378, in _shutdown_workers
    self.worker_result_queue.get()
  File "/home/hiwi/anaconda3/lib/python3.6/multiprocessing/queues.py", line 337, in get
    return _ForkingPickler.loads(res)
  File "/home/hiwi/anaconda3/lib/python3.6/site-packages/torch/multiprocessing/reductions.py", line 151, in rebuild_storage_fd
    fd = df.detach()
  File "/home/hiwi/anaconda3/lib/python3.6/multiprocessing/resource_sharer.py", line 57, in detach
    with _resource_sharer.get_connection(self._id) as conn:
  File "/home/hiwi/anaconda3/lib/python3.6/multiprocessing/resource_sharer.py", line 87, in get_connection
    c = Client(address, authkey=process.current_process().authkey)
  File "/home/hiwi/anaconda3/lib/python3.6/multiprocessing/connection.py", line 494, in Client
    deliver_challenge(c, authkey)
  File "/home/hiwi/anaconda3/lib/python3.6/multiprocessing/connection.py", line 722, in deliver_challenge
    response = connection.recv_bytes(256)        # reject large message
  File "/home/hiwi/anaconda3/lib/python3.6/multiprocessing/connection.py", line 216, in recv_bytes
    buf = self._recv_bytes(maxlength)
  File "/home/hiwi/anaconda3/lib/python3.6/multiprocessing/connection.py", line 407, in _recv_bytes
    buf = self._recv(4)
  File "/home/hiwi/anaconda3/lib/python3.6/multiprocessing/connection.py", line 379, in _recv
    chunk = read(handle, remaining)
ConnectionResetError: [Errno 104] Connection reset by peer
Traceback (most recent call last):


  File "<ipython-input-1-0cb739d0d1fb>", line 1, in <module>
    runfile('/home/hiwi/Desktop/HIWI_Data/Combustion_NN_Model/train.py', wdir='/home/hiwi/Desktop/HIWI_Data/Combustion_NN_Model')

  File "/home/hiwi/anaconda3/lib/python3.6/site-packages/spyder_kernels/customize/spydercustomize.py", line 668, in runfile
    execfile(filename, namespace)

  File "/home/hiwi/anaconda3/lib/python3.6/site-packages/spyder_kernels/customize/spydercustomize.py", line 108, in execfile
    exec(compile(f.read(), filename, 'exec'), namespace)

  File "/home/hiwi/Desktop/HIWI_Data/Combustion_NN_Model/train.py", line 70, in <module>
    main(config, args.resume)

  File "/home/hiwi/Desktop/HIWI_Data/Combustion_NN_Model/train.py", line 44, in main
    trainer.train()

  File "/home/hiwi/Desktop/HIWI_Data/Combustion_NN_Model/base/base_trainer.py", line 79, in train
    result = self._train_epoch(epoch)

  File "/home/hiwi/Desktop/HIWI_Data/Combustion_NN_Model/trainer/trainer.py", line 54, in _train_epoch
    output = self.model(data)

  File "/home/hiwi/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 477, in __call__
    result = self.forward(*input, **kwargs)

  File "/home/hiwi/Desktop/HIWI_Data/Combustion_NN_Model/model/model.py", line 31, in forward
    x = self.Fc1(x)

  File "/home/hiwi/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 477, in __call__
    result = self.forward(*input, **kwargs)

  File "/home/hiwi/anaconda3/lib/python3.6/site-packages/torch/nn/modules/linear.py", line 55, in forward
    return F.linear(input, self.weight, self.bias)

  File "/home/hiwi/anaconda3/lib/python3.6/site-packages/torch/nn/functional.py", line 1024, in linear
    return torch.addmm(bias, input, weight.t())

RuntimeError: Expected object of type torch.cuda.FloatTensor but found type torch.cuda.DoubleTensor for argument #4 'mat1'

The thing I am confused about is that, Pytorch model’s default data type is float then why it is giving this error?
and what does it mean by argument #4 'mat1' which is there in the error.

I also tried converting my data to double like this

train_in_data.double()
train_out_data.double
test_in_data.double()
test_out_data.double()
training_set = (train_in_data, train_out_data)
test_set = (test_in_data, test_out_data)

but it doesn’t

1 Like

It means you are allocating the model in a gpu but not the input

It is also being allocated on the GPU because the argument due to which I am getting error had torch.cuda type

Although this doesn’t seem to be the issue but just in case I also did that,

if training:
            self.dataset.train_labels.to('cuda')
            self.dataset.train_data.to('cuda')
if not  training:
            self.dataset.test_data.to('cuda')
            self.dataset.test_labels.to('cuda')

it doesn’t work then too

It looks like your data is of type torch.cuda.DoubleTensor while the model parameters are torch.cuda.FloatTensors.
I would recommend to cast your data to FloatTensors. If you really need double precision, you might also cast the model to double.

Hello,
Yeah I converted my model to double via model.double() and it is working alright now.

But just for curiosity, If I want to convert the data to float, how can I do that.
I am working on regression problem.
I am using sklearn.preprocessing.StandardScaler() for reading the data from csv files which converts by default into float64 aka double.
I also tried to convert the data into float by this:

train_in_data.float()
train_out_data.float()
test_in_data.float()
test_out_data.float()

but it doesn’t convert.

Can you suggest any other solution?
train_in_data, train_out_data, test_in_data, test_out_data are torch tensors.

Even after running the code which I mentioned above it says
torch.tensor(____, dtype=float64)

If you transform your tensors you should assign the result to them. Did you maybe forget it?
This should work:

train_in_data = train_in_data.float()
...

Sorry forgot to do that.
I tried it just like self.model.double().
It’s working now. Thank you for your time.

Hi, I had similar issue, but with the data, not the model.

Casting the data to float like that helped:

feats = feats.float()

The peculiar thing is that, although I had the tensor defined like that:

feats = torch.tensor(feats, dtype=torch.float64)

Torch still treated this object as a DoubleTensor, so these two ways of defining/casting to float are not equivalent. Pretty unintuitive, I would say

This is expected since float64 refers to double and thus a DoubleTensor. float32 represents a float and FloatTensor.