Conda keeps superseeding pytorch from cpu to gpu

I would suggest to try:

conda update pytorch

I prefer to always use this way because conda make thing messy easily. And with that you can see exactly wich packages and versions you will be updating or downloading.