How to switch to older version of pytorch?

Yes, you can specify the cudatookit version when installing the conda binaries.