Hi, I have the same error. Using PyTorch 1.7 on CUDA 11.0 installed thus:
pip install torch==1.7.0+cu110 torchvision==0.8.1+cu110 -f https://download.pytorch.org/whl/torch_stable.html
The error comes from line 138 on this script: https://github.com/MhLiao/DB/blob/master/assets/ops/dcn/functions/deform_conv.py
which calls the CUDA function in the compiled deform_conv_cuda.
I had to make some changes to the .cpp files back when upgrading to PyTorch 1.6, but had no problems after that. This is totally new.
Not sure if apex is being used?