Model is giving RuntimeError for CUDA but I mounted the data

Hello. The make the question title a bit more eloquent, basically I’m running a model with PyTorch tensors and for some reason every time I try to get the ouput from the model I get

*** RuntimeError: Expected object of device type cuda but got device cpu for argument #1 'self' in call to _th_index_select

Here’s the model I have:

class MatEmbedding(nn.Module):
    def __init__(self, config):
        self.config = config
        self.embedding_dim = self.config.embedding_dim
        self.num_embeddings = self.config.num_embeddings

        self.embed = nn.Embedding(num_embeddings=self.num_embeddings, embedding_dim=self.embedding_dim)

    def forward(self, inputs):
        indices = inputs[:, 0].to(torch.long)
        embed_result = self.embed(indices)

        return[embed_result, inputs[:, 1:]], dim=-1)

### Simple MLP model ###
class MLP(nn.Module):
    def __init__(self, config):
        self.config = config
        self.in_features = self.config.in_features
        self.out_features = self.config.out_features
        self.hidden_dim = self.config.hidden_dim
        self.embedding_dim = self.config.embedding_dim
        self.alpha = self.config.alpha

        self.embed = MatEmbedding(self.config)
        self.linear1 = nn.Linear(in_features=(self.in_features + self.embedding_dim - 1), out_features=self.hidden_dim)
        self.activ = nn.ELU(alpha=self.alpha)
        self.linear2 = nn.Linear(in_features=self.hidden_dim, out_features=self.out_features)

    def forward(self, inputs):
        out = self.embed(inputs)
        out = self.linear1(out)
        out = self.activ(out)
        out = self.linear2(out)

        return out

Regarding the data, when I run data.is_cuda I get True. Does anybody have any idea where I might be going wrong? Thanks in advance.

Could you run the code with CUDA_LAUNCH_BLOCKING=1 python and post the stack trace here? This would hopefully point to the line of code, which throws this error.
Also, could you add print(indices.device) before the self.embed(indices) call in MatEmbedding?

1 Like