After switching from pytorch 1.4.0 to 1.5.0 I noticed the CUDA_tensor_apply3 function is gone from CUDAApplyUtils.cuh, while the description of the latter says
The following CUDA_tensor_applyN (where N currently can be 1, 2, 3, or 4) functions apply a pointwise operator to N tensor(s).
Is this change intended? If so, what should I use instead of CUDA_tensor_apply3?
CUDA_tensor_apply3 was removed in favor of TensorIterator.
An example migration is shown in this PR.
Thank you. I stumbled into another problem: my fresh installation of pytorch lacks ATen/native/cuda folder, which contains headers with gpu_kernel function used together with TensorIterator in the example you provided. I installed pytorch via conda install -c pytorch pytorch torchvision --cudatoolkit=10.2.
Were you using CUDA_tensor_apply3 in a custom CUDA extension or what is your use case?
That’s right, I was using it when implementing custom activation functions. Often there is a substantial gain in performance compared to python version.