Hello,
I am attempting to train an LSTM on a specific GPU, but keep encountering weird behavior where the forward call adds data to both the device I intend (“cuda:1”), as well as the first available device (“cuda:0”). The following code snippet reproduces this behavior.
import torch
lstm = torch.nn.LSTM(input_size=100, hidden_size=200, batch_first=True, bidirectional=False).to(“cuda:1”)
data = torch.randn(10, 6, 100).to(“cuda:1”)
h0 = torch.randn(1, 10, 200).to(“cuda:1”)
c0 = torch.randn(1, 10, 200).to(“cuda:1”)
_ = lstm(data, (h0, c0))
print(torch.__version__)
1.9.0+cu102
As far as I know, I did not encounter this issue with previous versions of PyTorch, and I haven’t change my code which initializes and uses the LSTM.
EDIT - I tried version 1.8.0 and can confirm the exact same code only places data on the intended GPU.