And just after I post it I find a similar discussion: Are there any recommended methods to clone a model?
Edit: after reading through the other discussion I think its not what I am after. It focuses on how to properly copy a model and continue training with it. My use case is a little different as I want to properly clone the model and leave that cloning process in the computational graph so that I can differentiate all the way back to the original model.