Einsum, summation over ellipsis

I’m trying to use einsum in my implementation of Losses (eg. Dice Loss) for network training. So far I trained on 2D images, and it worked nicely, but now I tried to extend the code to work also on 3D data.
As I want the same code to work on both, 2 and 3D, I went for example from using
basically using ellipsis for the unknown number of dimensions. The desired output is still the same though - elementwise multiplication and then summation over all but first two dimensions.

But so, with this syntax, I want to do the summation over the ellipsis, which according to the documentation should work ( torch.einsum — PyTorch 1.8.1 documentation :
See note: " torch.einsum handles ellipsis (‘…’) differently from NumPy in that it allows dimensions covered by the ellipsis to be summed over, that is, ellipsis are not required to be part of the output. " ).

But for some reason it doesn’t work for me; when I use this particular operation on two tensors, both of size [32, 7, 24, 24, 24] for example, it throws an error: " RuntimeError: shape ‘[32, 7]’ is invalid for input of size 3096576 ". So it would seem like it doesn’t know how to sum over ellipsis after all?
Or am I somehow using it wrong?

I get the following results, and it seems to be working fine:

torch.einsum('bk..., bk... -> bk', torch.randn(32, 7, 24, 24, 24), torch.randn(32, 7, 24, 24, 24)).shape
Out[26]: torch.Size([32, 7])

Maybe you need to ensure your tensor’s shape.

Hi Eva (and Lart)!

Please check your pytorch version – this looks like an einsum()
bug that got fixed somewhere along the line. (I’m not sure exactly

Lart’s example works for me with a 1.8 nightly build, version
‘1.8.0.dev20210117’, but fails – giving your error – with an older
1.7 stable build, version ‘1.7.1’.

If you’re not using the current stable build (1.8.1), you might try
upgrading to that to see if things work. (I haven’t tried it on the
1.8.1 stable build, but the 1.8.0 nightly version suggests that the
stable build should work.)


K. Frank

Yes, you are right! :grinning_face_with_smiling_eyes: