hello everyone: when i use the transformer to extract information.
my src size is src = torch.size((368, 2, 256)), and tgt size is tgt = torch.size((768, 2, 256)). the last dimension is not suitable to transfomer embeding dimension requirements.(512). thus, use nn.linear to project 256 to 512.
However, it report issue. RuntimeError: the feature number of src and tgt must be equal to d_model. How to fix it? Thanks, best wishes
In the screenshot it seems you re passing nn.Embedding modules to the forward method of nn.Transformer, which is wrong, so I don’t think you are running this code.
In any case, based on the error message the d_model default value of 512 doesn’t fit the feature dimension and this should work:
trans = nn.Transformer(d_model=256, nhead=16, num_encoder_layers=12)
src = torch.randn(368, 2, 256)
tgt = torch.randn(768, 2, 256)
out = trans(src, tgt)
PS: you can post code snippets by wrapping them into three backticks ```, which would make debugging easier.
I am very grateful for your detailed guidance. Thanks. Best wishes