Getting unexpected NotImplementedError when running model

1

Hello. I’m currently working on debugging a model I wrote using PyTorch.

When I first ran output = model(data) I got the error

*** RuntimeError: Expected tensor for argument #1 'indices' to have scalar type Long; but got torch.cuda.DoubleTensor instead (while checking arguments for embedding)

No problem, I simply ran output = model(data.long()). Then I got a:

*** RuntimeError: Expected object of device type cuda but got device type cpu for argument #1 'self' in call to _th_index_select

I wasn’t sure what the error message exactly meant, but I recalled that I hadn’t put my model on the GPU device. So I ran:

model = model.to('cuda')
output = model(data.long())

but this time I got a:

*** NotImplementedError

This was a bit unexpected, and I checked if I had put in a NotImplementedError anywhere in the code but I hadn’t. I’m having some difficulty finding helpful resources, as I don’t even know what the causing piece of code is.

Would anybody have any idea how I should handle this issue? Just an FYI, these errors occurred while I was in the PDB interactive shell.

Thanks in advance.

2

  1. The .to() method needs the device code of the device, for gpus, the code is usually something like “cuda:0”. You can also use .cuda().

  2. Idk what device is data on, but calling .long() only changes it to Long type on its original device. If ur model is on gpu, u also have to change the inputs to gpu. In this case, I guess data is not on gpu.

3

As an addition to what @G.M said, could you please print the complete stack trace you get from the NotImplementedError?
You would get this error, e.g. if you have a typo in a method name (e.g. forwrad instead of forward) or if you use a wrong indentation.

4

@G.M I’ve also put the data on GPU via data.to('cuda'). I’ve never had a problem from not specifying the device and just using 'cuda'. Double checking right now data is a PyTorch tensor with device='cuda:0'.

5

Then could u please print the whole stack trace as @ptrblck said ?

6

Here’s the traceback that I’ve been able to get:

Traceback (most recent call last):
  File "./main.py", line 27, in <module>
    main()
  File "./main.py", line 20, in main
    output = model(data.long())
  File "C:\Users\user1\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 541, in __call__
    result = self.forward(*input, **kwargs)
  File "C:\Users\user1\Desktop\github\transformer_implementation\transformer.py", line 25, in forward
    enc_output = self.encoder(x)
  File "C:\Users\user1\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 541, in __call__
    result = self.forward(*input, **kwargs)
  File "C:\Users\user1\Desktop\github\transformer_implementation\encoder.py", line 54, in forward
    output = self.encoder_stacked(x)
  File "C:\Users\user1\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 541, in __call__
    result = self.forward(*input, **kwargs)
  File "C:\Users\user1\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 97, in forward
    raise NotImplementedError
NotImplementedError

I didn’t initially include too many details here in the original post, but the model that I’m using has a nn.ModuleList inside of it. After going through parts of the code, I found out that this was the problem. More specifically, it looks like:

self.encoder_stacked = nn.ModuleList([Encoder(self.config) for _ in range(self.N)])

According to this Stack Overflow question it seems that there are cases where the indentation may cause problems, but that’s not the case for me.

The Encoder module looks like this:

class Encoder(nn.Module):
    def __init__(self, config, dropout=True):
        super().__init__()
        self.config = config
        self.d_model = self.config.d_model

        self.attention = MultiheadAttention(self.config)

        if dropout:
            self.dropout = nn.Dropout(p=self.config.dropout_rate)
        else:
            self.dropout = nn.Dropout(p=0.0)

        self.norm = Normalization(features=self.d_model)
        self.ffnn = FeedforwardNN(self.config)

    def forward(self, x):
        attn_output = self.attention(x, x, x)
        sublayer1_output = self.norm(x + attn_output)
        sublayer1_output = self.dropout(sublayer1_output)

        ffnn_output = self.ffnn(sublayer1_output)
        sublayer2_output = self.norm(sublayer1_output + ffnn_output)
        sublayer2_output = self.dropout(sublayer2_output)

        return sublayer2_output
7

I think I may have found a solution here. I’ll do some more research based off of this answer.

8

Yes, ModuleList does not have a forward method. So u definitely can’t call on any ModuleList object as u did.