RuntimeError: Tensor for argument #2 'weight' is on CPU, but expected it to be on GPU (while checking arguments for cudnn_batch_norm) - Doubt

[ptrblck]

You could take a look at this post, which points to the CPU and CUDA implementations of the native convolutions.