RuntimeError: expected scalar type Long but found Float

1

Printing type shows it is a LongTensor still the error prevails. It is a character based model that i am building for the popular ‘dinosaur names’ problem. here is the code:

class din_model(nn.Module):
    def __init__(self, max_len):
        super(din_model, self).__init__()
        self.rnn = nn.RNN(1, 64)
        self.dense = nn.Linear(64,27)
    
    def forward(self, input_):
        reshape_ = input_.view((input_.size(0),input_.size(1),1))
        print(reshape_.type(), reshape_.size())
        rnn_, _ = self.rnn(reshape_)
        dense_ = self.dense(rnn_)
        return F.log_softmax(dense_, dim=1)
    
model = din_model(27)
optimizer = optim.Adam(model.parameters(), lr=0.01)
loss_fn = nn.NLLLoss()

for epoch in range(3): 
    for t,l in train_dataloader:
        output_batch = model(torch.tensor(t, dtype=torch.long))
        loss = loss_fn(output_batch, l)
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()
    print("Epoch is {}, loss is {}".format(epoch, loss.data))

the error is here:

RuntimeError                              Traceback (most recent call last)
<ipython-input-54-1717991d3fdc> in <module>
      1 for epoch in range(3):
      2     for t,l in train_dataloader:
----> 3         output_batch = model(torch.tensor(t, dtype=torch.long))
      4         loss = loss_fn(output_batch, l)
      5         # clear previous gradients, compute gradients of all variables wrt loss

/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
    725             result = self._slow_forward(*input, **kwargs)
    726         else:
--> 727             result = self.forward(*input, **kwargs)
    728         for hook in itertools.chain(
    729                 _global_forward_hooks.values(),

<ipython-input-53-e8e162814236> in forward(self, input_)
      8         reshape_ = input_.view((input_.size(0),input_.size(1),1))
      9         print(reshape_.type(), reshape_.size())
---> 10         rnn_, _ = self.rnn(reshape_)
     11         dense_ = self.dense(rnn_)
     12         return F.log_softmax(dense_, dim=1)

/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
    725             result = self._slow_forward(*input, **kwargs)
    726         else:
--> 727             result = self.forward(*input, **kwargs)
    728         for hook in itertools.chain(
    729                 _global_forward_hooks.values(),

/opt/conda/lib/python3.7/site-packages/torch/nn/modules/rnn.py in forward(self, input, hx)
    233         if batch_sizes is None:
    234             result = _impl(input, hx, self._flat_weights, self.bias, self.num_layers,
--> 235                            self.dropout, self.training, self.bidirectional, self.batch_first)
    236         else:
    237             result = _impl(input, batch_sizes, hx, self._flat_weights, self.bias,

RuntimeError: expected scalar type Long but found Float

the result for print(reshape_.type(), reshape_.size()) is torch.LongTensor torch.Size([32, 27, 1])

please if anyone can help me resolve it, that would be great.

2

Hi,

RNNs don’t support long tensors as input AFAIK. This should be a regular float Tensor torch.tensor(t, dtype=torch.float).

4 Likes
3

this saved me so much time. thanks a ton. Why would they throw an exact opposite error message is beyond me.

2 Likes
4

Because it depends in which order you do the op. As the high level rnn does not check that directly but its constituent functions do.
And in these functions, the first arg is usually the input given by the user. And we expect everything to be of the same time as it. Hence the error you see.

1 Like
5

Oh i see. thanks for the explanation around this. I am assuming you are referring to _impl function where weights and bias are coming in as float? if thats the case, how are LSTMs any different?

1 Like
6

All these functions have custom c++ implementation that are not really normalized (mostly because some of them need to be dispatched to special backend like mkl/cudnn and other are just implemented by us). So maybe the order of the arguments is different for them.
Or also there are some type checks that happen at a different moment.

Not sure really :confused:

1 Like
7

I think the message must be :

RuntimeError: expected scalar type Float but found Long :thinking:

8

Well it depends which argument goes where haha
If you do a + b or b + a you will get flipped messages. These messages always assume that the first argument has the “correct” type and the second one is wrong.