If your input always have the same size, you should enable it all the time.
But it will influence which algorithm cudnn is using only while the flag is enabled. So setting it during training does not influence inference in any way.
Hello. I would like to ask a few questions about the behavior of torch.backends.cudnn.benchmark = True.
Does the mini-batch size matter? Many people say that benchmarking uses the same cache if image input size is the same. However, I have not found a clear explanation of whether changing batch size is OK.
How many caches can it manage? For example, I might have two types of input: 224x224 and 320x320. Would changing between the two types of images constantly require additional benchmarking or would there be two separate caches?
hello guys, I have a quick question about the torch.backends.cudnn.benchmark = True
When you say the input_size cannot change, does that apply to each convolution layer?
I have a UNet design using dense blocks. Since in a block, input for each layer is different, does that mean I cannot use torch.backends.cudnn.benchmark = True ?
Is there any workaround for dense block so that I can use torch.backends.cudnn.benchmark = True ?