# Why is torch.mm non deterministic? **URL:** \ **Category:** Uncategorized\ **Created:** [July 28, 2022, 2:09pm UTC](https://discuss.pytorch.org/t/why-is-torch-mm-non-deterministic/157735 "2022-07-28T14:09:42Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![Megh\_Bhalerao](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/megh_bhalerao/32/13547_2.png) [@Megh\_Bhalerao](https://discuss.pytorch.org/u/Megh_Bhalerao)\ **Post date:** [July 28, 2022, 2:09pm UTC](https://discuss.pytorch.org/t/why-is-torch-mm-non-deterministic/157735/1 "2022-07-28T14:09:42Z")
Hello, I get the following error (this error has been referenced many times on the internet, but putting it here anyways) - ```auto Traceback (most recent call last): File "main.py", line 79, in main() File "main.py", line 69, in main distillation = Distillation(train_loader, train_dataset, model_wt_path, config_dict, args.seed) # passing the mean and stddev of the dataset for the images, this is more specific to images when done in this context File "/x0/megh98/projects/ddist/distillation.py", line 44, in __init__ train_acc = self.run_validation(self.train_loader) File "/x0/megh98/projects/ddist/distillation.py", line 66, in run_validation logits = self.net(img) File "/x0/megh98/anaconda3/envs/ddist/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl return forward_call(*input, **kwargs) File "/x0/megh98/projects/ddist/models/networks.py", line 67, in forward out = self.classifier(out) File "/x0/megh98/anaconda3/envs/ddist/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl return forward_call(*input, **kwargs) File "/x0/megh98/anaconda3/envs/ddist/lib/python3.8/site-packages/torch/nn/modules/linear.py", line 103, in forward return F.linear(input, self.weight, self.bias) RuntimeError: Deterministic behavior was enabled with either `torch.use_deterministic_algorithms(True)` or `at::Context::setDeterministicAlgorithms(true)`, but this operation is not deterministic because it uses CuBLAS and you have CUDA >= 10.2. To enable deterministic behavior in this case, you must set an environment variable before running your PyTorch application: CUBLAS_WORKSPACE_CONFIG=:4096:8 or CUBLAS_WORKSPACE_CONFIG=:16:8. For more information, go to https://docs.nvidia.com/cuda/cublas/index.html#cublasApi_reproducibility ``` when I set - `torch.use_deterministic_algorithms(True)`. This is no surprise. When I went to the docs here - [torch.use\_deterministic\_algorithms — PyTorch 1.12 documentation](https://pytorch.org/docs/stable/generated/torch.use_deterministic_algorithms.html) , it mentions that `torch.mm` is a non deterministic operation. So, I have the following 2 questions - 1. Why would a simple matrix multiplication such as `torch.nn` be a non-deterministic operation? 2. From the error traceback it seems like the non-determinism is coming from the `F.linear` function, which after looking at the pytorch GitHub seems to use torch.matmul which inturn uses torch.mm atleast according to my understanding - and this is what is leading to the non determinism. Am I right in my understanding here? Thanks! ---
**Author:** ![smth](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/smth/32/13_2.png) [@smth](https://discuss.pytorch.org/u/smth)\ **Post date:** [July 28, 2022, 9:48pm UTC](https://discuss.pytorch.org/t/why-is-torch-mm-non-deterministic/157735/2 "2022-07-28T21:48:53Z")
the reasoning is given in both the error message and the documentation and it points to this link: [cuBLAS :: CUDA Toolkit Documentation](https://docs.nvidia.com/cuda/cublas/index.html#cublasApi_reproducibility) ---
**Author:** ![Megh\_Bhalerao](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/megh_bhalerao/32/13547_2.png) [@Megh\_Bhalerao](https://discuss.pytorch.org/u/Megh_Bhalerao)\ **Post date:** [July 28, 2022, 10:19pm UTC](https://discuss.pytorch.org/t/why-is-torch-mm-non-deterministic/157735/3 "2022-07-28T22:19:34Z")
Thanks @smth - yes, I did read the explanation in the link earlier, but it seems like the explanation is a general one where they mention stuff about cuda streams etc. But, I wanted to know what is specific about the `torch.mm` function that makes it non-deterministic, is it related the explanation given here - [cuBLAS :: CUDA Toolkit Documentation](https://docs.nvidia.com/cuda/cublas/index.html#batching-kernels) - is it something related to the parallelism of the matrix multiplies? ---
**Author:** ![smth](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/smth/32/13_2.png) [@smth](https://discuss.pytorch.org/u/smth)\ **Post date:** [July 28, 2022, 10:20pm UTC](https://discuss.pytorch.org/t/why-is-torch-mm-non-deterministic/157735/4 "2022-07-28T22:20:51Z")
yes, the non determinism comes from torch.mm calling cublas and cublas being non deterministic in these cases ---
**Author:** ![Megh\_Bhalerao](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/megh_bhalerao/32/13547_2.png) [@Megh\_Bhalerao](https://discuss.pytorch.org/u/Megh_Bhalerao)\ **Post date:** [August 29, 2022, 4:40am UTC](https://discuss.pytorch.org/t/why-is-torch-mm-non-deterministic/157735/5 "2022-08-29T04:40:17Z")
Hi, sorry to open this issue again, but I am still not completely clear as to why torch.mm is non deterministic. I understand that there are some backend cuda operations that are non deterministic, but a matrix multiply must be deterministic right? Other wise if I do AB where A and B are matrices, and I do the same operation again, I will get 2 different results which does not make sense? Am I missing some crucial point here? Please let me know and thanks again for your time. ---
**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\ **Post date:** [August 29, 2022, 7:36am UTC](https://discuss.pytorch.org/t/why-is-torch-mm-non-deterministic/157735/6 "2022-08-29T07:36:26Z")
Depending on the algorithm and its implementation a matmul kernel could be non-deterministic. Especially if you are using workspaces to stored intermediate results, algorithms using split-k, etc.