Transformer dimension probelem

hello everyone: when i use the transformer to extract information.
my src size is src = torch.size((368, 2, 256)), and tgt size is tgt = torch.size((768, 2, 256)). the last dimension is not suitable to transfomer embeding dimension requirements.(512). thus, use nn.linear to project 256 to 512.

However, it report issue. RuntimeError: the feature number of src and tgt must be equal to d_model. How to fix it? Thanks, best wishes


In the screenshot it seems you re passing nn.Embedding modules to the forward method of nn.Transformer, which is wrong, so I don’t think you are running this code.
In any case, based on the error message the d_model default value of 512 doesn’t fit the feature dimension and this should work:

trans = nn.Transformer(d_model=256, nhead=16, num_encoder_layers=12)
src = torch.randn(368, 2, 256)
tgt = torch.randn(768, 2, 256)

out = trans(src, tgt)

PS: you can post code snippets by wrapping them into three backticks ```, which would make debugging easier.

I am very grateful for your detailed guidance. Thanks. Best wishes