Dimension out of range

1

Dear all,

we’re experimenting with Transformers for deep-learning the translation from Protein to DNA sequences. Here’s an excerpt of our code:

##
## ITERATOR
##
BATCH_SIZE = 10
train_iterator, test_iterator = BucketIterator.splits(
    (train_data, test_data), 
    batch_size = BATCH_SIZE,
    device = device, sort = False)

##
## TRANSFORMER
##
model = nn.Transformer(d_model=1, nhead=1)
criterion = torch.nn.CrossEntropyLoss()
optimizer = torch.optim.Adam(model.parameters(), lr=1)
for i, batch in enumerate(train_iterator):
    
    lbl = batch.seq_id
    src = batch.prot_seq
    trg = batch.dna_seq

    optimizer.zero_grad()
    model.train()

    output = model(src, trg)
    print(model)

However, this code produces the following error:

Traceback (most recent call last):
File “learner.py”, line 195, in
output = model(src, trg)
File “/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/torch/nn/modules/module.py”, line 550, in call
result = self.forward(*input, **kwargs)
File “/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/torch/nn/modules/transformer.py”, line 117, in forward
print(f’src.size(2) --> {src.size(2)}’)
IndexError: Dimension out of range (expected to be in range of [-2, 1], but got 2)

src and trg both are mxn matrices, where m the length of the protein and DNA sequence respectively and n is the BATCH_SIZE. According to the documentation (https://pytorch.org/docs/master/generated/torch.nn.Transformer.html), the third dimensions of src and trg must be the the feature number E. What does this exactly mean? What’s the required form of src and trg? Also, how can I add E to the src and the trg matrices?

Thanks for shedding light into my confusion!