LSTM module: 'tuple' object has no attribute 'dim' :

Hi All,

I am trying to modify this example Link for pytorch, Though I am getting the same error, as discussed here Link .But I have passed the correct dimension:

My model is like below:

model = torch.nn.Sequential(
    torch.nn.Linear(1,20),
    torch.nn.LSTM(input_size = 20, hidden_size = 20,num_layers = 1,bidirectional = False),
    torch.nn.Linear(20, 1),
)

And I’m trying to predict the output by passing the X_train, where X_train is the 3D vector of size (XX,49,1)

y_pred = model(X_train_) # this line gives the error,

#Here is the complete code

import numpy as np
import matplotlib.pyplot as plt
import pandas
import math,os
import pickle
import torch
from torch.autograd import Variable


from sklearn.metrics import explained_variance_score


df = pandas.read_csv('./data/household_power_consumption', sep=';',na_values=['?']) # data loading

df = df.dropna()# Remove 

limit_rows = 150
df = df[:limit_rows]
# Get the header info
df.columns.values.tolist()
# get global active power,vlotage,intensity
data = df[['Global_active_power','Voltage','Global_intensity']]

temp = data[['Global_active_power','Voltage']]

power = list(temp['Global_active_power'].get_values().flatten())
voltage = list(temp['Voltage'].get_values().flatten())

sequence = 50
result = []
for index in range(len(power) - sequence):
    result.append(power[index: index + sequence])
result = np.array(result)
    

result = np.array(result)  # shape (2000, 50)


row = int(round(0.8 * result.shape[0]))
train = result[:row, :]
np.random.shuffle(train)
X_train = train[:, :-1] # get the first sequence-1 values, this is input to
y_train = train[:, -1]# get the last value , this is target value

# X_train is my array of dimension xx*49, where xx depend on the size of data points

X_train_ = np.reshape(X_train, (X_train.shape[0], X_train.shape[1], 1))

X_train_ = Variable(torch.from_numpy(X_train_),requires_grad=True).float()
y_train = Variable(torch.from_numpy(y_train),requires_grad=False).float()

X_test = result[row:, :-1]
y_test = result[row:, -1]

X_test_ = np.reshape(X_test, (X_test.shape[0], X_test.shape[1], 1))

X_test_ = Variable(torch.from_numpy(X_test_),requires_grad=False).float()
y_test = Variable(torch.from_numpy(y_test),requires_grad=False).float()

model = torch.nn.Sequential(
    torch.nn.Linear(1,20),
    torch.nn.LSTM(input_size = 20, hidden_size = 20,num_layers = 1,bidirectional = False),
    torch.nn.Linear(20, 1),
)
loss_fn = torch.nn.MSELoss(size_average=False)

learning_rate = 1e-4
optimizer = torch.optim.Adam(model.parameters(), lr=learning_rate)
for t in range(10):
    # Forward pass: compute predicted y by passing x to the model.
    y_pred = model(X_train_)

    # Compute and print loss.
    loss = loss_fn(y_pred, y_test)
    #print(t, loss.data[0])

    optimizer.zero_grad()

    loss.backward()

    optimizer.step()

Can anyone suggest, what I am doing wrong?

Here is the traceback: ()

File "<ipython-input-125-080466b19e66>", line 1, in <module>
    runfile('/home/saurabh/Documents/power_predict.py', wdir='/home/saurabh/Documents')

  File "/home/saurabh/anaconda2/lib/python2.7/site-packages/spyder/utils/site/sitecustomize.py", line 705, in runfile
    execfile(filename, namespace)

  File "/home/saurabh/anaconda2/lib/python2.7/site-packages/spyder/utils/site/sitecustomize.py", line 94, in execfile
    builtins.execfile(filename, *where)

  File "/home/saurabh/Documents/power_predict.py", line 73, in <module>
    y_pred = model(X_train_)

  File "/home/saurabh/anaconda2/lib/python2.7/site-packages/torch/nn/modules/module.py", line 357, in __call__
    result = self.forward(*input, **kwargs)

  File "/home/saurabh/anaconda2/lib/python2.7/site-packages/torch/nn/modules/container.py", line 67, in forward
    input = module(input)

  File "/home/saurabh/anaconda2/lib/python2.7/site-packages/torch/nn/modules/module.py", line 357, in __call__
    result = self.forward(*input, **kwargs)

  File "/home/saurabh/anaconda2/lib/python2.7/site-packages/torch/nn/modules/linear.py", line 55, in forward
    return F.linear(input, self.weight, self.bias)

  File "/home/saurabh/anaconda2/lib/python2.7/site-packages/torch/nn/functional.py", line 833, in linear
    if input.dim() == 2 and bias is not None:

AttributeError: 'tuple' object has no attribute 'dim'

LSTM outputs a tuple containing output and hidden state.
Your second Linear layer tries to take this tuple as input.

Thanks for the reply @jpeg729.
So to solve that issue, I need to index out the LSTM first output and feed it to the second layer, while I am moving forward in the network,
but how can I do with the model(X_train_) directly ? as I, can’t do anything with the model declaration, as indexing with in the model declaration is not allowed.

You can’t do it with nn.Sequential, but you can do it by subclassing nn.Module and writing a custom .forward method as shown in many tutorials.