Floating-point masks are not supported for fast path

1

I am trying to enable the fastpath to speed up the inference in the “class MultiheadAttention()” then I hit this.pytorch/torch/nn/modules/activation.py at 042d764872a6307ed2f087cf8a741b2106370207 · pytorch/pytorch · GitHub “floating-point masks are not supported for fast path” . my atten_mask is float. does that mean we have to convert it to int?

2

You should pass it as a binary mask (i.e. a BoolTensor). From the docs:

Binary and float masks are supported. For a binary mask, a True value indicates that the
corresponding position is not allowed to attend.