Continuing the discussion from Converting Multi-output model from Keras to Pytorch results in worse results:
You’ve unfortunately deleted the old topic.
Would you like to repost the original question here? 
I found the answer in another post. My problem was with loading the data. I was basically training on a subset of the data.