Hi!

I am writing a neural network in pytorch and I am running into some issues. I am implementing an attention layer, however I get a runtime error when I want to repeat my hidden layer. My model is as follows:

class Model_P(nn.Module):

def **init**(self, hidden_size, input_size):

super(Model_P, self).**init**()

self.hidden_size = hidden_size

self.input_size = input_size

self.lstm = nn.LSTM(input_size, hidden_size, 1, bias=False, batch_first=True, bidirectional=True)

self.dropout = nn.Dropout(0.5)

self.softsign_1 = nn.Softsign()

self.attention_hidden_vector = nn.Linear(hidden_size + 2 * hidden_size, hidden_size)

self.attention_scoring_fn = nn.Linear(hidden_size, 1, bias=False)

self.dense = nn.Linear(hidden_size,1)

self.relu_1 = nn.ReLU()

```
def forward(self, x):
#model en data parameters must be the same, thus convert to float
x.float()
output, hidden = self.lstm(x.float())
output = self.softsign_1(output.float())
src_len = output.shape[0]
hidden = hidden.repeat(src_len, 1, 1)
attn_hidden = torch.tanh(self.attn_hidden_vector(torch.cat((hidden, output), dim=2)))
attn_scoring_vector = self.attn_scoring_fn(attn_hidden).squeeze(2)
attn_scoring_vector = attn_scoring_vector.permute(1, 0)
output = F.softmax(attn_scoring_vector, dim=1)
output = self.dense(output)
output = self.relu_1(output)
return output
```

I get the following error: AttributeError: ‘tuple’ object has no attribute ‘repeat’

I have already tried to convert the tuple ‘hidden’ to a tensor, by converting the tuple to an numpy array, and then creating a tensor, via the following code:

hidden = np.asarray(hidden)

hidden = torch.from_numpy(hidden)

However this raises other erros, which I have not succeeded to solve. Does anybody know of a solution? Many thanks in advance!