# What does torch.backends.cudnn.benchmark do? **URL:** \ **Category:** Uncategorized\ **Created:** [August 8, 2017, 4:21pm UTC](https://discuss.pytorch.org/t/what-does-torch-backends-cudnn-benchmark-do/5936 "2017-08-08T16:21:17Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![John\_Zhang](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/john_zhang/32/45177_2.png) [@John\_Zhang](https://discuss.pytorch.org/u/John_Zhang)\ **Post date:** [August 8, 2017, 4:21pm UTC](https://discuss.pytorch.org/t/what-does-torch-backends-cudnn-benchmark-do/5936/1 "2017-08-08T16:21:17Z")
whats the difference when setting it True or False? ---
**Author:** ![albanD](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/alband/32/215_2.png) [@albanD](https://discuss.pytorch.org/u/albanD)\ **Post date:** [August 8, 2017, 4:42pm UTC](https://discuss.pytorch.org/t/what-does-torch-backends-cudnn-benchmark-do/5936/2 "2017-08-08T16:42:37Z")
This flag allows you to enable the inbuilt cudnn auto-tuner to find the best algorithm to use for your hardware. ---
**Author:** ![fmassa](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/fmassa/32/58_2.png) [@fmassa](https://discuss.pytorch.org/u/fmassa)\ **Post date:** [August 8, 2017, 4:43pm UTC](https://discuss.pytorch.org/t/what-does-torch-backends-cudnn-benchmark-do/5936/3 "2017-08-08T16:43:13Z")
It enables benchmark mode in cudnn. benchmark mode is good whenever your input sizes for your network do not vary. This way, cudnn will look for the optimal set of algorithms for that particular configuration (which takes some time). This usually leads to faster runtime. But if your input sizes changes at each iteration, then cudnn will benchmark every time a new size appears, possibly leading to worse runtime performances. ---
**Author:** ![yxchng](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/y/7bcc69/32.png) [@yxchng](https://discuss.pytorch.org/u/yxchng)\ **Post date:** [August 31, 2017, 1:47am UTC](https://discuss.pytorch.org/t/what-does-torch-backends-cudnn-benchmark-do/5936/4 "2017-08-31T01:47:40Z")
I tried using it with torchvision’s resnet101 but it gives worse performance. Is it normal? @fmassa @albanD ---
**Author:** ![fmassa](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/fmassa/32/58_2.png) [@fmassa](https://discuss.pytorch.org/u/fmassa)\ **Post date:** [September 1, 2017, 9:32am UTC](https://discuss.pytorch.org/t/what-does-torch-backends-cudnn-benchmark-do/5936/5 "2017-09-01T09:32:42Z")
It depends on the task. If your input size is changing a lot, then it might hurt runtime, if not, it should be much faster. ---
**Author:** ![mjchen611](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/mjchen611/32/1526_2.png) [@mjchen611](https://discuss.pytorch.org/u/mjchen611)\ **Post date:** [September 9, 2017, 3:06am UTC](https://discuss.pytorch.org/t/what-does-torch-backends-cudnn-benchmark-do/5936/6 "2017-09-09T03:06:46Z")
whats the difference when setting cudnn.enabled True or False? ---
**Author:** ![tstandley](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/tstandley/32/3534_2.png) [@tstandley](https://discuss.pytorch.org/u/tstandley)\ **Post date:** [February 17, 2018, 7:30am UTC](https://discuss.pytorch.org/t/what-does-torch-backends-cudnn-benchmark-do/5936/7 "2018-02-17T07:30:59Z")
Does it work to turn it on for training (where I have a constant input size) and turn it off for validation (where my input size isn’t constant)? Do I just set the constant before doing my validation? ---
**Author:** ![albanD](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/alband/32/215_2.png) [@albanD](https://discuss.pytorch.org/u/albanD)\ **Post date:** [February 19, 2018, 2:11pm UTC](https://discuss.pytorch.org/t/what-does-torch-backends-cudnn-benchmark-do/5936/8 "2018-02-19T14:11:19Z")
Yes, it will work to change the value! ---
**Author:** ![Alpha](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/alpha/32/25271_2.png) [@Alpha](https://discuss.pytorch.org/u/Alpha)\ **Post date:** [March 16, 2018, 8:01am UTC](https://discuss.pytorch.org/t/what-does-torch-backends-cudnn-benchmark-do/5936/9 "2018-03-16T08:01:22Z")
I have a question. e.g. I have a **net1** whose input sizes don’t vary. And I have other nn.Module named **net\_Loss** whose input sizes vary. I only need to optimeze **net1’s** parameters. So should I use the cudnn.benchmark _True or False_? Thank you ! ---
**Author:** ![salihkaragoz](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/salihkaragoz/32/3802_2.png) [@salihkaragoz](https://discuss.pytorch.org/u/salihkaragoz)\ **Post date:** [April 20, 2018, 7:10am UTC](https://discuss.pytorch.org/t/what-does-torch-backends-cudnn-benchmark-do/5936/10 "2018-04-20T07:10:30Z")
Hello, IMHO, if your networks connect each other and if net1 is first network, you should use cudnn.benchmark = True. and also you ve mentioned I only need net1’s parameters. I think you should use. ---
**Author:** ![DeepLearner17](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/deeplearner17/32/3185_2.png) [@DeepLearner17](https://discuss.pytorch.org/u/DeepLearner17)\ **Post date:** [April 20, 2018, 11:36am UTC](https://discuss.pytorch.org/t/what-does-torch-backends-cudnn-benchmark-do/5936/11 "2018-04-20T11:36:10Z")
Hello, @albanD `When we set cudnn.benchmark=True` 1. How can l get access to the whole family of algorithms that potentially can be executed? (display them) 2. How can l print the cudnn algorithm run at each iteration ? Thank you ---
**Author:** ![albanD](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/alband/32/215_2.png) [@albanD](https://discuss.pytorch.org/u/albanD)\ **Post date:** [April 20, 2018, 12:45pm UTC](https://discuss.pytorch.org/t/what-does-torch-backends-cudnn-benchmark-do/5936/12 "2018-04-20T12:45:07Z")
1. I’m afraid you can’t. That would depend on the operation you’re performing. For example, for forward pass of convolution, you can find the list [here](https://github.com/pytorch/pytorch/blob/1848cad10802db9fa0aa066d9de195958120d863/aten/src/ATen/native/cudnn/Conv.cpp#L486-L494). 2. You would have to add this print directly in the cpp code linked above. ---
**Author:** ![Diego](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/diego/32/6989_2.png) [@Diego](https://discuss.pytorch.org/u/Diego)\ **Post date:** [May 8, 2018, 3:37pm UTC](https://discuss.pytorch.org/t/what-does-torch-backends-cudnn-benchmark-do/5936/13 "2018-05-08T15:37:44Z")
Can you show an example on how to use this line? Can you place it anywhere in the code? Or before the forward pass? ---
**Author:** ![ruotianluo](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ruotianluo/32/383_2.png) [@ruotianluo](https://discuss.pytorch.org/u/ruotianluo)\ **Post date:** [July 6, 2018, 9:24pm UTC](https://discuss.pytorch.org/t/what-does-torch-backends-cudnn-benchmark-do/5936/17 "2018-07-06T21:24:29Z")
Can I have part of the inference to be benchmarked. Like this: ```auto x = func1(x) x = x[:20] torch.backends.cudnn.benchmark = True x = func2(x) torch.backends.cudnn.benchmark = False ``` ---
**Author:** ![PabloRR100](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/pablorr100/32/5891_2.png) [@PabloRR100](https://discuss.pytorch.org/u/PabloRR100)\ **Post date:** [October 4, 2018, 4:01pm UTC](https://discuss.pytorch.org/t/what-does-torch-backends-cudnn-benchmark-do/5936/18 "2018-10-04T16:01:05Z")
What if you are training different networks in the same script exposed to the same input sizes? I am training a single ResNet44 and after that an ensemble of 3 ResNets 18 on CIFAR10. How do I manage the `cudnn.benchmark = True`? First at the beggining of the ResNet44 training and then once again after the beggining of the first ResNet18? Thanks! ---
**Author:** ![kuzand](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/kuzand/32/8268_2.png) [@kuzand](https://discuss.pytorch.org/u/kuzand)\ **Post date:** [January 17, 2019, 10:08am UTC](https://discuss.pytorch.org/t/what-does-torch-backends-cudnn-benchmark-do/5936/19 "2019-01-17T10:08:45Z")
> [@fmassa](#): > > benchmark mode is good whenever your input sizes for your network do not vary. Hi. Can you please clarify what do you mean by “input size”? Is it the image size, like 224? Since the input size of network is always fixed and the images are resized to the same size before inputing them to network, in which cases it can vary? ---
**Author:** ![fabianjul](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/f/a587f6/32.png) [@fabianjul](https://discuss.pytorch.org/u/fabianjul)\ **Post date:** [March 15, 2019, 1:04pm UTC](https://discuss.pytorch.org/t/what-does-torch-backends-cudnn-benchmark-do/5936/20 "2019-03-15T13:04:29Z")
When you have a fully convolutional network ---
**Author:** ![Oli](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/oli/32/9773_2.png) [@Oli](https://discuss.pytorch.org/u/Oli)\ **Post date:** [April 29, 2019, 6:33pm UTC](https://discuss.pytorch.org/t/what-does-torch-backends-cudnn-benchmark-do/5936/21 "2019-04-29T18:33:46Z")
I find that `torch.backends.cudnn.benchmark` increases the speed for my YOLOv3 model by a lot, like 30-40%. Furthermore, it lowers the memory footprint after it completes the benchmark. It even works when my input images vary in size between each batch, neat! I was thinking about having the network optimize on a few smaller `torch.randn(...)` to benchmark on, and then start the training. I hope that this could allow me to increase the batch size since the memory footprint is lower after the bechmark. What do you guys thing? ---
**Author:** ![Shubhankar](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/s/b9bd4f/32.png) [@Shubhankar](https://discuss.pytorch.org/u/Shubhankar)\ **Post date:** [January 13, 2020, 5:55pm UTC](https://discuss.pytorch.org/t/what-does-torch-backends-cudnn-benchmark-do/5936/23 "2020-01-13T17:55:47Z")
Is this still the only way to do it? ---
**Author:** ![albanD](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/alband/32/215_2.png) [@albanD](https://discuss.pytorch.org/u/albanD)\ **Post date:** [January 13, 2020, 6:02pm UTC](https://discuss.pytorch.org/t/what-does-torch-backends-cudnn-benchmark-do/5936/24 "2020-01-13T18:02:14Z")
Hi, I don’t think this has changed I’m afraid. But there might have been changes to this code so not sure. [Next page](https://discuss.pytorch.org/t/what-does-torch-backends-cudnn-benchmark-do/5936.md?page=2)