I assume it’s specific to FlashAttentionV2, but also am not using Windows and thus cannot verify the support of previous implementations.
Based on e.g. this comment it seems to be the case.
I assume it’s specific to FlashAttentionV2, but also am not using Windows and thus cannot verify the support of previous implementations.
Based on e.g. this comment it seems to be the case.