Hi!
Is there a way to check if torch is using deterministic algorithms after I have set torch.use_deterministic_algorithms(True)?
Naively, I would have assumed that at least torch.backends.cudnn.deterministic is set to True after this call to reflect that convolutions are using deterministic algorithms.
It depends on your knowledge about the used algorithms, but you could e.g. profile the code via Nsight Systems and check which kernels are used as some kernels are known to be (non-)deterministic. I’m not aware of any other way to “check” it besides setting the variable and assuming the framework respects this setting.
Thank you very much for the confirmation!
I think that it may be valuable in some cases, especially for debugging, to have the means to check if the used modules are set to be deterministic.
I will add it as a feature request.
I assume you made a feature request and it was implemented, because now there is a way to check. For others who stumble across this thread, the answer is:
torch.are_deterministic_algorithms_enabled()
https://pytorch.org/docs/stable/generated/torch.are_deterministic_algorithms_enabled.html