AttributeError: 'tuple' object has no attribute 'repeat'

nlp
1

I am doing an assignment on Neural Machine Translation and training an LSTM specifically . During Decoding stage I got this Error " AttributeError: ‘tuple’ object has no attribute ‘repeat’ ". I am unable to understand what the error (that I am getting) is, and how I can solve it?

Traceback (most recent call last):
File “train.py”, line 618, in
early_stopping=early_stopping)
File “train.py”, line 186, in train_epochs
decode(model, dataset, epoch)
File “/home/hisrar/NMT/kg_model.py”, line 357, in beam_decode
dec_hidden.repeat(1, batch_size * beam_size, 1)
AttributeError: ‘tuple’ object has no attribute ‘repeat’

Part of the program where I get the Error is

    batch_size, beam_size = self.config.batch_size, self.config.beam_size

    # [1, batch_size x beam_size]

    dec_input = torch.ones(1, batch_size * beam_size,
                           dtype=torch.long,
                           device=self.device) * SOS_ID

   #[num_layers, batch_size * beam_size, hidden_size]

    #dec_hidden = dec_hidden.repeat(1, beam_size,1)
  
    if enc_outputs is not None:
        enc_outputs = enc_outputs.repeat(1, beam_size, 1)
        enc_length = enc_length.repeat(beam_size)

screenshot
screenshot955×689 48.8 KB

2

the attribute repeat is available to Tensor but the variable dec_hidden is of type Tuple. Make sure to convert your tuple to a Tensor and then it’ll work. If it’s a Tuple of 1 object do,

dec_hidden[0].repeat(1, beam_size, 1)
3 
dec_hidden[0].repeat(1, beam_size, 1)

changing dec_hidden to dec_hidden[0], I got a new error.

File “/usr/local/lib/python3.5/dist-packages/torch/nn/modules/rnn.py”, line 187, in check_hidden_size
raise RuntimeError(msg.format(expected_hidden_size, tuple(hx.size())))
RuntimeError: Expected hidden[0] size (2, 64, 500), got (64, 500)

4

can you print what dec_hidden is?

5

dec_hidden
dec_hidden1234×406 57.9 KB

6

Ok so it seems that that object is a Tuple of 2 tensors. Do you want to repeat both Tensors or just one? If that have the same shape you can use torch.cat then that’ll convert your tuple to a Tensor that can then use .repeat

7

Yes I want to repeat both Tensors.

8

Could try something like this?

dec_hidden_repeeat = [x.repeat(1, beam_size, 1) for x in dec_hidden]

9

Tried the Idea with little variation, and it works well . @AlphaBetaGamma96 Thank you so much for your Help

1 Like