Embedding became non-deterministic in PyTorch 2.11

1

Hi!

Upgrading PyTorch to newer one (2.8 => 2.12), I noticed that some of the models became non-deterministic.

After some debugging, searching through the code etc I found this PR: reland D89141291 add fused compute_grad_weight kernel when segments are few but large (landed in PyTorch 2.11 if I got it right)

From my understanding of it, after this, nn.Embedding backwards path became non-deterministic in some cases (depending on shapes).

A couple of questions that I’m trying to figure out:

  • Is it overall an expected change in terms of determinism?
  • Is it expected to have some detailed toggle for it in the future releases? Does it make sense opening an issue/PR for it? Right now it seems like it’s only can be disabled by the overall torch.use_deterministic_algorithms(true), which also impacts a lot of other algorithms
  • Should it be documented in torch.use_deterministic_algorithms? It isn’t mentioned there right now.
2

Hi,

Thank you very much for reporting this!
We do not guarantee that ops specific implementations will remain deterministic or not by default indeed.

Only when the use_deterministic_algorihtms(True) flag is set do we provide that guarantee. The document should be updated indeed! Thanks for flagging it, preparing a PR for that.

Right now it seems like it’s only can be disabled by the overall torch.use_deterministic_algorithms(true), which also impacts a lot of other algorithms

I would be curious about that use case! In particular, why would you want embedding to be deterministic but not other ops? The overall run will not be determistic if you do that?

Also note that you can set this flag temporarily during the backward to only use such algorithms during that time.

3

There it is Add deterministic mode test for embedding backward and use DeterministicGuard by albanD · Pull Request #187682 · pytorch/pytorch · GitHub