# Torch.svd is slow in GPU compared to CPU **URL:** \ **Category:** Uncategorized\ **Created:** [December 4, 2017, 6:58pm UTC](https://discuss.pytorch.org/t/torch-svd-is-slow-in-gpu-compared-to-cpu/10770 "2017-12-04T18:58:48Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![Sabra\_Ossen](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/sabra_ossen/32/1666_2.png) [@Sabra\_Ossen](https://discuss.pytorch.org/u/Sabra_Ossen)\ **Post date:** [December 4, 2017, 6:58pm UTC](https://discuss.pytorch.org/t/torch-svd-is-slow-in-gpu-compared-to-cpu/10770/1 "2017-12-04T18:58:48Z")
In the following graph, you can see the execution time in seconds taken for Torch.svd in both CPU and GPU. As you can see the time taken for GPU is larger than that time taken for CPU and keeps increasing as we increase the number of records. Please note that the GPU memory has a size of 12 GB and the maximum number of records that can be run with svd is 3.25 million. ![chart (4)](https://discuss.pytorch.org/uploads/default/original/2X/0/0e51cb5abb53e6d4d6e238f9a76d3122cd30be54.png) Following is the code I used. ``` import torch from datetime import datetime import sys n_records = sys.argv[1]; x = torch.zeros(n_records, 300).cuda() s = datetime.now() u, _, _ = torch.svd(x, some=True) e = datetime.now() del x el = (e-s).total_seconds() print(el) ``` Following is the output from NVIDIA visual profiler for 1000 and 1 million records respectively. As you can see after the ‘kernelPointWiseApply2’ cuda kernel invocation there is a certain amount of idle time that is introduced. This idle time between the first and second kernel functions in the “compute” section has also increased. ![svd_1K_time](https://discuss.pytorch.org/uploads/default/original/2X/f/f291bfe6e68a7b27dffccb07d7d856ad54301564.png) ![svd_1M_time](https://discuss.pytorch.org/uploads/default/original/2X/c/c92301115a5c5f6d495c2a543616e2d44c02b6bf.png) Could anyone help me in understanding why this idle time occurs and why it keeps increasing as we increase the size of the dataset? ---
**Author:** ![SimonW](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/simonw/32/1702_2.png) [@SimonW](https://discuss.pytorch.org/u/SimonW)\ **Post date:** [December 4, 2017, 7:06pm UTC](https://discuss.pytorch.org/t/torch-svd-is-slow-in-gpu-compared-to-cpu/10770/2 "2017-12-04T19:06:17Z")
This is because magma’s implementation of ?gesvd is used under the hood. Unfortunately, the implementation is not purely on GPU and still calls LAPACK/BLAS functions ([https://github.com/maxhutch/magma/blob/master/src/dgesvd.cpp](https://github.com/maxhutch/magma/blob/master/src/dgesvd.cpp)). That is probably why you are seeing these idle times on GPU. ---
**Author:** ![Sabra\_Ossen](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/sabra_ossen/32/1666_2.png) [@Sabra\_Ossen](https://discuss.pytorch.org/u/Sabra_Ossen)\ **Post date:** [December 5, 2017, 1:59am UTC](https://discuss.pytorch.org/t/torch-svd-is-slow-in-gpu-compared-to-cpu/10770/3 "2017-12-05T01:59:31Z")
So my understanding here is that PyTorch uses a modified version of magma (which includes more BLAS) and not the original [magma](http://icl.cs.utk.edu/magma/software/index.html). Where in the code is the magma function actually invoked, is it [https://github.com/pytorch/pytorch/blob/master/aten/src/THC/generic/THCTensorMathMagma.cu#L273](https://github.com/pytorch/pytorch/blob/master/aten/src/THC/generic/THCTensorMathMagma.cu#L273)? I would like to get an idea of the path of invocation for SVD. ---
**Author:** ![SimonW](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/simonw/32/1702_2.png) [@SimonW](https://discuss.pytorch.org/u/SimonW)\ **Post date:** [December 5, 2017, 4:16am UTC](https://discuss.pytorch.org/t/torch-svd-is-slow-in-gpu-compared-to-cpu/10770/4 "2017-12-05T04:16:04Z")
No. magma calls LAPACK/BLAS. PyTorch doesn’t modify it. There are several LAPACK calls in `?gesdd` of magma [https://github.com/maxhutch/magma/blob/master/src/sgesdd.cpp](https://github.com/maxhutch/magma/blob/master/src/sgesdd.cpp). Sorry I was linking the wrong file. And yes, the link you provided is part of the execution path. ---
**Author:** ![Sabra\_Ossen](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/sabra_ossen/32/1666_2.png) [@Sabra\_Ossen](https://discuss.pytorch.org/u/Sabra_Ossen)\ **Post date:** [December 5, 2017, 10:42pm UTC](https://discuss.pytorch.org/t/torch-svd-is-slow-in-gpu-compared-to-cpu/10770/5 "2017-12-05T22:42:59Z")
Sorry, I think you might have misunderstood the first line of my reply. What I meant was that as I understood this ([http://icl.cs.utk.edu/magma/software/index.html](http://icl.cs.utk.edu/magma/software/index.html)) is the magma original library. And this ([https://github.com/maxhutch/magma/blob/master/src/sgesdd.cpp](https://github.com/maxhutch/magma/blob/master/src/sgesdd.cpp)) is a fork of magma to include more BLAS. The second is what PyTorch uses. Is that correct? And yes from what I see in the file you pointed out this is not a pure GPU implementation, there are several calls to LAPACK. Thanks very much for the clarification. ---
**Author:** ![SimonW](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/simonw/32/1702_2.png) [@SimonW](https://discuss.pytorch.org/u/SimonW)\ **Post date:** [December 5, 2017, 10:45pm UTC](https://discuss.pytorch.org/t/torch-svd-is-slow-in-gpu-compared-to-cpu/10770/6 "2017-12-05T22:45:13Z")
Oh I see. Sorry about that. I just took the first google result and didn’t look at the owner name 😃 I believe we use the original magma. ---
**Author:** ![Sabra\_Ossen](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/sabra_ossen/32/1666_2.png) [@Sabra\_Ossen](https://discuss.pytorch.org/u/Sabra_Ossen)\ **Post date:** [December 5, 2017, 11:31pm UTC](https://discuss.pytorch.org/t/torch-svd-is-slow-in-gpu-compared-to-cpu/10770/7 "2017-12-05T23:31:48Z")
If so I believe this should be the file ([https://bitbucket.org/icl/magma/src/d646183b0ca94c48ebb8aa1ce75869e98766a605/src/dgesdd.cpp?at=default&fileviewer=file-view-default](https://bitbucket.org/icl/magma/src/d646183b0ca94c48ebb8aa1ce75869e98766a605/src/dgesdd.cpp?at=default&fileviewer=file-view-default)). I wasn’t able to find a sgesdd.cpp in the magma repo ([https://bitbucket.org/icl/magma/src/d646183b0ca9/src/?at=default](https://bitbucket.org/icl/magma/src/d646183b0ca9/src/?at=default)). ---
**Author:** ![linyu](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/l/ebca7d/32.png) [@linyu](https://discuss.pytorch.org/u/linyu)\ **Post date:** [May 9, 2018, 1:02pm UTC](https://discuss.pytorch.org/t/torch-svd-is-slow-in-gpu-compared-to-cpu/10770/8 "2018-05-09T13:02:50Z")
Hi, do you solve the problem of GPU is slower than CPU. I encounter the same issue when using the function torch.gesv. Waiting for your apply, thank you! ---
**Author:** ![chenhao1umbc](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/chenhao1umbc/32/14836_2.png) [@chenhao1umbc](https://discuss.pytorch.org/u/chenhao1umbc)\ **Post date:** [August 6, 2019, 1:35am UTC](https://discuss.pytorch.org/t/torch-svd-is-slow-in-gpu-compared-to-cpu/10770/9 "2019-08-06T01:35:34Z")
I got the same issue here… temp.shape Out[14]: torch.Size([1140, 1140]) which is low rank structure u,s,v = torch.svd(temp) it took 3.5 seconds! vs u,s,v = torch.svd(temp.cpu()) it took 0.23 seconds! ridiculously slow here… ---
**Author:** ![CadyWang](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/c/35a633/32.png) [@CadyWang](https://discuss.pytorch.org/u/CadyWang)\ **Post date:** [February 1, 2020, 7:12am UTC](https://discuss.pytorch.org/t/torch-svd-is-slow-in-gpu-compared-to-cpu/10770/10 "2020-02-01T07:12:11Z")
Hello, have you solved this problem?I also encountered, do not know how to solve