My project has an implementation of a transformer sequence-to-sequence model. When loading it for the first time, we get this UserWarning
: enable_nested_tensor is True, but self.use_nested_tensor is False because encoder_layer.norm_first was True
. This is not a very helpful warning to me because 1) I don’t immediately see why that’s a problem and 2) there’s no call to action: what am I supposed to do about it?
2 Likes
Any news on this?
I get the same warning
set batch_first=True in nn.TransformerEncoderLayer(**kwargs)
Oddly, this didnt resolve my problem even though everyone online claims it does. Must be a skill issue on my part…
It definitely sounds like that warning can be improved… feel free to file an issue about this.