RuntimeError: input must have 3 dimensions, got 2 LSTM

Also one last question. Why are you just passing it the entire dataset? Why are you not breaking it into batches?

Wow. That fixes this original issue. Thank you.

I passed in the whole dataset because I don’t know the benefits of passing the data in batches. I have the data split into a train and test set.

Another question though. I get the error RuntimeError: expected scalar type Double but found Float. I tried adding .double() to self.lstm = nn.LSTM(53, self.hidden_size, self.num_layers, batch_first=True) and x = x.unsqueeze(0), but I still get the error. Do you know anoy solutions to this problem?

Hmm. Can I see the code where you do .double()? Also have you tried adding it to the hidden state?

Here is what I have now:

class TeacherUpdated(nn.Module):
    def __init__(self):
        # python requires to call ancestor's initilizer manually!
        super(TeacherUpdated, self).__init__()
        self.num_layers = 3
        self.hidden_size = 200
        self.lstm = nn.LSTM(53, self.hidden_size, self.num_layers, batch_first=True).double()
        self.linear = nn.Linear(200, 2).double()

    def init_state(self, batch_size):
        return (torch.zeros(self.num_layers, batch_size, self.hidden_size),
                torch.zeros(self.num_layers, batch_size, self.hidden_size))

    def forward(self, x):
        x = x.unsqueeze(0).double()
        print(x.size())
        hs = self.init_state(1)
        x, hs = self.lstm(x, hs)
        x = x.reshape(-1, 200)
        x = self.linear(x)
        return x

If I add .double() to hs = self.init_state(1), I get a different error.

Which line is the error on?

x, hs = self.lstm(x, hs)

In the init_state function can you try setting each torch.zeros to a double?

Wow, I tried that before and it did not work, but it works now all of a sudden. Thank you, that resolved that issue.

Sorry for all the questions, but I get another error now: n <module> weight = named_parameters['{}.weight'.format(layer_name)] KeyError: 'layer1.weight'. I originally had the model with a nn.Linear before switching to an LSTM and i’m guessing this error has to do something with the new LSTM.

        for layer_name in ['layer1', 'layer2', 'layer3']:
            with torch.no_grad():
                weight = named_parameters['{}.weight'.format(layer_name)]
                bias = named_parameters['{}.bias'.format(layer_name)].unsqueeze(1)
                weight_bias = torch.cat((weight, bias),dim=1)
                norm = torch.norm(weight_bias, dim=1, keepdim=True).add_(1e-6)
                clip_coef = norm.reciprocal_().mul_(max_norm).clamp_(max=1.0)
                weight.mul_(clip_coef)
                bias.mul_(clip_coef)

Yes there is no layer 1 in a lstm. I am not totally sure how to do it with an lstm. You may have to mess around with printing the parameters of your model to figure it out. But the layer names you currently have are wrong.

I have looked online and I am not able to find the name of the layers. Do you know how I can print the names of them? Or is there an alternative to doing this?

Sorry for the late response but you can do something like this

for name in model.named_parameters():
     print(name[0])

to print out all the names of the weights and biases in your model.

Thanks. That makes sense. Currently, I have that code segment commented out and it seems to work fine without it. I will probably add that back with help from this code.

Thanks again for all your help.

1 Like

Hello, could you post the full code please?