F.conv2d with large tensor。Tried to allocate 81.00 GiB

[smth]

@Zhang_Jiguo while you cannot set the CuDNN algorithm, you can limit the max Workspace size that CuDNN is allowed to take to compute the convolution. That should be sufficient for your purpose.

You can set this workspace size using the environment variable CUDNN_CONV_WSCAP_DBG=4096, where for example 4096 here specifies 4096 megabytes.
In your code, you can specify this env variable before the import torch code.
For example:

import os
os.environ["CUDNN_CONV_WSCAP_DBG"] = 4096
import torch

Alternatively, you can specify it on the command-line:

CUDNN_CONV_WSCAP_DBG=4096 python your_script.py

References:

8.7 GB CUDA block allocated and then freed by Conv2d forward

Thanks for the update! It’s defined by the available algorithm and depends on the memory layout, data type, memory alignment etc. No, I don’t know how much performance will be lost, as benchmarking is not even working. I think the right approach would be to limit the cudnn workspace size requirements and skip algos, with a workspace > threshold requirement. If you are using cudnn>=8.0.5, you could use this env variable as a workaround for now: CUDNN_CONV_WSCAP_DBG=4096 python script.py ar…

Provide a mechanism to limit the workspace size of cudnn convolution