To(device) not working for hidden variable

Hi I have this piece of codes where I want to pass the input and hidden variable to a LSTM cell, and I tried to move hidden variable h to cuda device because of error “Input and hidden tensors are not at the same device, found input tensor at cuda:0 and hidden tensor at cpu” :

def _run_rnn_packed(self, cell, x, x_lens, h=None):
R.pack_padded_sequence(x,x_lens,batch_first=True).to(self.device)
if h is not None:
print(“hidden variabe device before to device : {}”.format(h.is_cuda))
h.to(self.device)
print(“hidden variabe device after to device : {}”.format(h.is_cuda))
output, h = cell(x_packed, h)
else:
output, h = cell(x_packed)

but “h.to(device)” doesn’t seem to work. The second print statement still gave me false. Even when I did “h.cuda()”, the tensor is still not on GPU, Any idea how to fix it?

Are you initializing the hidden state yourself? If you don’t manually initialize the hidden state, PyTorch will auto-initialize them for you (all zeros). This way, you wouldn’t have to mess around with shifting the hidden to cuda.

Side note: There’s an option in the editor called “Preformatted text”. Please use that to format your code to make it more readable. :slight_smile:

1 Like

Hi,

The .to() operation on Tensors is not inplace, you need to do h = h.to(self.device).

2 Likes

The hidden state is passed from another layer so I didn’t use the default initialization. But I found the reason.

Btw thanks for the side note I didn’t notice the “Preformatted text” button. Will do next time!

Thanks that’s also what I need.