Double values not recognized as such

1

I am trying to build a minimal working example in PyTorch using an LSTM. I use a mock example that predicts for a sequence of 3 time steps one out of 4 labels.

I get an error message which says
Expected object of scalar type Double but got scalar type Float for argument #2 ‘mat2’ in call to _th_mm

I am not sure what is missing. I assume if you are more familiar with PyTorch problem is easy to spot but my understanding is that my data is already double (i.e. float64=double, no?) Can someone give me a pointer what is wrong?

from torch.utils.data import DataLoader,Dataset
import random
class MyDataset(Dataset):
    
    def __init__(self, data, labels):
        self.data=data
        self.labels=labels
        
    def __getitem__(self, index):
        return self.data[index], self.labels[index]
    
    def __len__(self):
        return len(self.data)
    
mock_data=torch.tensor([
                            [[.3, .2], [.2,.57], [.93,.9]],
                            [[.33, .23], [.25,.57], [.63,.29]],
                            [[.63, .4], [.25,.9], [.23,.29]],
                            [[.343, .223], [.125,.57], [.163,.29]],
                            [[.3, .2], [.2,.57], [.93,.9]],
                            [[.33, .23], [.25,.57], [.63,.29]],
                            [[.63, .4], [.25,.9], [.23,.29]],
                            [[.343, .223], [.125,.57], [.163,.29]],
                        ]).double()
mock_label=torch.tensor([1, 2, 0, 3, 2, 2, 1, 0]).long()
MAX_LABELS=4

ds = MyDataset(mock_data, mock_label)
dl = DataLoader(ds, shuffle=True, batch_size=4)

import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim

class TheNet(nn.Module):
    def __init__(self, dim_x):
        super(TheNet, self).__init__()
        self.lstm = nn.LSTM(input_size=dim_x, hidden_size=64, batch_first=True)
        self.out = nn.Linear(64, MAX_LABELS)
       
    def forward(self, data):
        lstm_out, _ = self.lstm(data)
        out = self.out(lstm_out.view(len(data), -1))
        label_score = F.log_softmax(out, dim=1)
        return label_score
    
model = TheNet(2)
loss_function = nn.NLLLoss()
optimizer = optim.SGD(model.parameters(), lr=0.1)

for epoch in range(300):  
    for seq, label in dl:
        model.zero_grad()
        print(seq.shape)
        scores = model(seq)
        break
#         loss = loss_function(scores, labels)
#         loss.backward()
#         optimizer.step()
2

Hi,

The problem is that your model is created in type float by default. So if you need it to be of a different type, you need to change it with: model.double(). after you created it.

Another thing, you cannot use .view() directly on the output of the lstm (because of the way the lstm returns its output), so you will get an error. You can use .reshape() here instead.

You will also get a size mismatch. between the output of the lstm and your linear. But most likely because you use the output of the 3 timesteps (192 features) and not only one (64 features).

1 Like
3

Ah, thank you! You are right about the view part, too but I was not even getting that far.