I have a tensor of shape 12X9X1024. I have created 9 individual fully connected layers which take 12 X 1024 shape array as input. Here 12 is the batch size and 1024 is the input dimension to the fully connected layer.
I planned to append the output of each FCN to a list. Then convert the list of tensors to a tensor using torch.cat. When the code is at out.append(f(tensors[:,i,:]).view(bs,-1,2))
I get this error RuntimeError: Tensor for 'out' is on CPU, Tensor for argument #1 'self' is on CPU, but expected them to be on GPU (while checking arguments for addmm)
class Classifier(nn.Module):
def __init__(self,hidden_dim):
super().__init__()
_,centers = config.get_inds()
_,counts = np.unique(np.array(centers, dtype= np.int32),return_counts=True)
self.fcn = []
self.classes = sum(counts)
for j in counts:
self.fcn.extend([nn.Linear(hidden_dim,2*j)])
def forward(self,tensors):
"""
tensor: n_batch X n_center X hidden_dim
"""
bs = tensors.shape[0]
out = []
for i,f in enumerate(self.fcn):
out.append(f(tensors[:,i,:]).view(bs,-1,2))
out_tensor = torch.Tensor((bs,self.classes,2),device=tensors.device,requires_grad=True)
torch.cat(out,dim=1,out=out_tensor)
out_tensor = F.log_softmax(out_tensor, dim=1)
return out_tensor
How to make sure the list of tensors are on GPU and has requires_grad=True?