Can't convert a given np.ndarray to a tensor

1 
import torch
from torch import nn
from torch.autograd import Variable
import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
xy = pd.read_csv("all.csv",names=['pm2.5','pm10,s02','no2','co','o3','tem','hig_tem','low_tem','rain','qiya','shidu','zuixiaoshidu','fengsu_10min','fengsu','rizuidafengsu','rizhao'])
x_data = xy.iloc[6:-1,:-1].values
y_data = xy.iloc[:6,:-1].values


x_data = Variable(torch.from_numpy(x_data))
y_data = Variable(torch.from_numpy(y_data))
.....

I want to run an RNN but i can’t solve the data problem . Then i got an error:

RuntimeError: can't convert a given np.ndarray to a tensor - it has an invalid type. The only supported types are: double, float, int64, int32, and uint8.

how can i make it available?

1 Like
2

Can you print the type of your data? you might have to cast it as a one of the accepted types before you convert?

3 
print(x_data[:5,:5])

result:
[[-164.2 -211.7 -87.2 -25.8 -1272.1]
[-27.8 4.1 34.0 6.6 -381.1]
[64.5 111.9 129.7 30.8 788.7]
[170.9 241.6 44.3 36.4 2754.0]
[-259.6 -403.2 -207.5 -65.5 -3982.6]]

4

Seeing floats doesn’t necessarily mean it is a float. What does x_data.dtype return? Try and explicitly cast x_data as a type. x_data.astype(dtype = 'float32)

1 Like
5

the x_data.dtype is object and i try

x_data.astype(np.float32)
x_data = Variable(torch.from_numpy(x_data))

but i failed
get this result:

ValueError: could not convert string to float: '#VALUE!'
6

basically you need to remove all the rows that contain #value!. the data set you are using doesn’t contain only numbers as I expected. Once you have removed these values it should work. Also try and make sense of the error messages… sometimes they are not very helpful but in this case it is.

7

thanks very much
i will try again

8

ooooops
i try to make data small and accurate

xy = pd.read_csv("all.csv",names=['pm2.5','pm10,s02','no2','co','o3','tem','hig_tem','low_tem','rain','qiya','shidu','zuixiaoshidu','fengsu_10min','fengsu','rizuidafengsu','rizhao'])
x_data = xy.iloc[7:16,:2192].values
y_data = xy.iloc[:6,:2192].values

x_data.astype(np.float32)
x_data = Variable(torch.from_numpy(x_data))

result:

RuntimeError: can't convert a given np.ndarray to a tensor - it has an invalid type. The only supported types are: double, float, int64, int32, and uint8.

and i have checked the data …there is no #VALUE

9

Could you please print np.unique(x_data) before passing it to the tensor constructor?
As @Tank explained there seem to be some invalid values in your array.

1 Like
10

the result of np.unique(x_data)

TypeError: '>' not supported between instances of 'float' and 'str'

but i don’t know where the str?

11

you probably have some sort of nan value or empty string e.g. ’ ’ somewhere. Trying np.unique() on each column or row of the dataset might help you narrow down the problem.

12

I had a similar problem when converting data read from pandas to PyTorch.
I was able to solve it using pd.to_numeric(...)

Hence, for the code at the top, the fix could be:

x_data = pd.to_numeric(xy.iloc[6:-1,:-1].values)
y_data = pd.to_numeric(xy.iloc[:6,:-1].values)