Why does he use Variable here?

Sorry I’m very new to deep learning and PyTorch. I’m looking into some codes about LSTM with pytorch.

Why does he use Variable in

dataX = Variable(torch.Tensor(np.array(x)))
dataY = Variable(torch.Tensor(np.array(y)))

trainX = Variable(torch.Tensor(np.array(x[0:train_size])))
trainY = Variable(torch.Tensor(np.array(y[0:train_size])))

testX = Variable(torch.Tensor(np.array(x[train_size:len(x)])))
testY = Variable(torch.Tensor(np.array(y[train_size:len(y)])))

and

def forward(self, x):
        h_0 = Variable(torch.zeros(
            self.num_layers, x.size(0), self.hidden_size))
        
        c_0 = Variable(torch.zeros(
            self.num_layers, x.size(0), self.hidden_size))

According to this question, python - How to load a list of numpy arrays to pytorch dataset loader? - Stack Overflow, you should use TensorDataset to convert a list of 2d arrays into pytorch inputs, why does he use Variable here?

Besides, why does he use Variable in forward function? Is torch.zeros not enough?

The second question is that, when I try to run this code, I got this error, ‘RuntimeError: Input and parameter tensors are not at the same device, found input tensor at cuda:0 and parameter tensor at cpu’. How should I fix this? Which lines should I add .to(device) command?

I added .cuda() as follows:

dataX = Variable(torch.Tensor(np.array(x))).cuda()
dataY = Variable(torch.Tensor(np.array(y))).cuda()

trainX = Variable(torch.Tensor(np.array(x[0:train_size]))).cuda()
trainY = Variable(torch.Tensor(np.array(y[0:train_size]))).cuda()

testX = Variable(torch.Tensor(np.array(x[train_size:len(x)]))).cuda()
testY = Variable(torch.Tensor(np.array(y[train_size:len(y)]))).cuda()

and

def forward(self, x):
        h_0 = Variable(torch.zeros(
            self.num_layers, x.size(0), self.hidden_size)).cuda()
        
        c_0 = Variable(torch.zeros(
            self.num_layers, x.size(0), self.hidden_size)).cuda()

but still got this error.

Don’t use the Variable class anymore, as it’s deprecated since PyTorch 0.4 (we are about to release 1.12.0 soon so 0.4 is “quite old” :slight_smile: ).

torch.zeros would work in newer PyTorch releases and you should just remove the usage of Variable entirely from the script.

Make sure all (input) tensors as well as the model parameters are on the same device.

1 Like

Thank you very much! I fixed all the forementioned problems.

The training and testing data are 3d tensors with the first dimension being sample size. Does that mean I can directly put in a 3d tensor with the first dimension being sample size without using DataLoader as inputs?

Because I’m more familiar with keras, where you can directly put in 3d arrays as inputs with first dimension being sample size.

Yes, you can directly execute the forward pass if you pass the input tensor in the expected shape to the model. By default the batch dimension would be dim0, but be careful about RNNs as they use dim1 as the default dimension for the batch size (you could use batch_first=True to change this behavior, but check the docs for these layers for more information).

The DataLoader will batch the samples for you and is a convenient method to use a Dataset which loads a single sample only without worrying about batching, shuffling etc.

1 Like