# Copy.deepcopy() vs clone() **URL:** \ **Category:** Uncategorized\ **Created:** [September 3, 2019, 7:33am UTC](https://discuss.pytorch.org/t/copy-deepcopy-vs-clone/55022 "2019-09-03T07:33:29Z")\ **Posts on this page:** 17\ **Page:** 1
**Author:** ![Shisho\_Sama](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/shisho_sama/32/6926_2.png) [@Shisho\_Sama](https://discuss.pytorch.org/u/Shisho_Sama)\ **Post date:** [September 3, 2019, 7:33am UTC](https://discuss.pytorch.org/t/copy-deepcopy-vs-clone/55022/1 "2019-09-03T07:33:29Z")
when copying modules/tensors around, which one should I use? are they interchangable? Thanks a lot ---
**Author:** ![spanev](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/spanev/32/14892_2.png) [@spanev](https://discuss.pytorch.org/u/spanev)\ **Post date:** [September 3, 2019, 9:19am UTC](https://discuss.pytorch.org/t/copy-deepcopy-vs-clone/55022/2 "2019-09-03T09:19:14Z")
Hi @Shisho_Sama, For `Tensor`s in most cases, you should go for `clone` since this is a PyTorch operation that will be recorded by autograd. ```auto >>> t = torch.rand(1, requires_grad=True) >>> t.clone() tensor([0.4847], grad_fn=) # <=== as you can see here ``` When it comes to `Module`, there is no `clone` method available so you can either use `copy.deepcopy` or create a new instance of the model and just copy the parameters, as proposed in this post [Deep copying PyTorch modules](https://discuss.pytorch.org/t/deep-copying-pytorch-modules/13514/2). ---
**Author:** ![Shisho\_Sama](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/shisho_sama/32/6926_2.png) [@Shisho\_Sama](https://discuss.pytorch.org/u/Shisho_Sama)\ **Post date:** [September 3, 2019, 10:53am UTC](https://discuss.pytorch.org/t/copy-deepcopy-vs-clone/55022/3 "2019-09-03T10:53:21Z")
Hi, Thanks a lot. So this means, when I do clone() on tensors, their clones will still be on the graph and any operations on them will be reflected in the graph right? for example changing the values or attributes will also change the original tensor as well? or affect the graph computation when doing backward pass? In the case of the following two code snippets, what does happen in each case? I think, deepcopy disregards any graph related information and just copies the data as if it is a simple object while the clone, will create a new tensor which any operations on it will be reflected in the graph and in order to prevent this I need to use detach as well. am I right? ```auto weights_encoder = sae_model.encoder[0].weight.data.clone() weights_decoder = sae_model.decoder[0].weight.data.clone() ``` or ```auto weights_encoder = copy.deepcopy(sae_model.encoder[0].weight.data) weights_decoder = copy.deepcopy(sae_model.decoder[0].weight.data) ``` ---
**Author:** ![spanev](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/spanev/32/14892_2.png) [@spanev](https://discuss.pytorch.org/u/spanev)\ **Post date:** [September 3, 2019, 12:11pm UTC](https://discuss.pytorch.org/t/copy-deepcopy-vs-clone/55022/4 "2019-09-03T12:11:05Z")
When you use `.data`, you get a new `Tensor` with `requires_grad=False`, so cloning it won’t involve autograd. So both are equivalent, but there might be a (small) speed difference, I am not sure about that. Another use case could is when you want to clone/copy a non-parameter `Tensor` without autograd. You should use `.detach()` (and [not data](https://github.com/pytorch/pytorch/issues/6990#issuecomment-384680164)) before cloning: ```auto >>> t = torch.rand(1, requires_grad=True) >>> t.detach().clone() tensor([0.4847]) ``` ---
**Author:** ![Shisho\_Sama](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/shisho_sama/32/6926_2.png) [@Shisho\_Sama](https://discuss.pytorch.org/u/Shisho_Sama)\ **Post date:** [September 3, 2019, 12:25pm UTC](https://discuss.pytorch.org/t/copy-deepcopy-vs-clone/55022/5 "2019-09-03T12:25:14Z")
Thank you very much. I really appreciate it 🙂 ---
**Author:** ![dio\_din](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/dio_din/32/21505_2.png) [@dio\_din](https://discuss.pytorch.org/u/dio_din)\ **Post date:** [April 24, 2020, 12:42pm UTC](https://discuss.pytorch.org/t/copy-deepcopy-vs-clone/55022/6 "2020-04-24T12:42:08Z")
Is there any difference with `t.clone().detach()`? ---
**Author:** ![bobvo23](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/b/f475e1/32.png) [@bobvo23](https://discuss.pytorch.org/u/bobvo23)\ **Post date:** [May 7, 2020, 7:56pm UTC](https://discuss.pytorch.org/t/copy-deepcopy-vs-clone/55022/7 "2020-05-07T19:56:52Z")
Yes there is. Though both methods create same outcomes, however, t.clone().detach() is less efficient. The t.clone() with create a copy that attaches to the graph, then it will create another copy (detach()). So there will be more redundant. ---
**Author:** ![pinocchio](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/pinocchio/32/22093_2.png) [@pinocchio](https://discuss.pytorch.org/u/pinocchio)\ **Post date:** [June 16, 2020, 8:57pm UTC](https://discuss.pytorch.org/t/copy-deepcopy-vs-clone/55022/8 "2020-06-16T20:57:02Z")
> [@spanev](#): > > For `Tensor` s in most cases, you should go for `clone` since this is a PyTorch operation that will be recorded by autograd. I never understood this. Why would one ever want to have clone be in the computation graph? It’s just the identity! When I make a copy of something I usually expect a brand new object, with new memory allocation and new instance of the object class it belongs. Not just copying pointers/references around. Can you clarify? ---
**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\ **Post date:** [June 17, 2020, 7:38am UTC](https://discuss.pytorch.org/t/copy-deepcopy-vs-clone/55022/9 "2020-06-17T07:38:52Z")
Answered [here](https://discuss.pytorch.org/t/confusion-about-using-clone/39673/12) and [here](https://discuss.pytorch.org/t/clone-and-detach-in-v0-4-0/16861/21). ---
**Author:** ![pinocchio](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/pinocchio/32/22093_2.png) [@pinocchio](https://discuss.pytorch.org/u/pinocchio)\ **Post date:** [June 17, 2020, 7:01pm UTC](https://discuss.pytorch.org/t/copy-deepcopy-vs-clone/55022/10 "2020-06-17T19:01:16Z")
Let me see if I understand (it seems the accepted answer here is outdated, `.data` is not in the library or going to be removed according to what I’ve read in other answers with from albanD). `.clone()` produces a new tensor instance with a new memory allocation to the tensor data. In addition it remembers the history of the original tensor and is connected to the earlier graph and appears as `CloenBackward`. The main advantage it seems is that its safer wrt in-place ops afaik. `deepcopy` make a deep copy of the original tensor meaning it creates a new tensor instance with a new memory allocation to the tensor data (it definitively does this part correctly from my tests). I assume it also does a complete copy of the history too, either pointing to the old history or create a brand new deep copy history. I’m unsure how to test this but I believe if it is to behave as a proper deep copy method then it should create a new history that is a mirror of the earlier (instead of just pointing to it). Test I did wrt memory allocation: ```auto def clone_vs_deepcopy(): import copy import torch x = torch.tensor([1,2,3.]) x_clone = x.clone() x_deep_copy = copy.deepcopy(x) # x.mul_(-1) print(f'x = {x}') print(f'x_clone = {x_clone}') print(f'x_deep_copy = {x_deep_copy}') ``` output ```auto x = tensor([-1., -2., -3.]) x_clone = tensor([1., 2., 3.]) x_deep_copy = tensor([1., 2., 3.]) ``` since neither changed it must be a different memory. I just realized I could have checked it with `id` or something…alas. I am still seeking clarification on the history part. Is it a deep copy of that or a pointer copy if we use deep copy? I know for know for clone it is a pointer copy to the original history and not a complete deep copy. * * * related: > ---
**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\ **Post date:** [June 18, 2020, 5:49am UTC](https://discuss.pytorch.org/t/copy-deepcopy-vs-clone/55022/11 "2020-06-18T05:49:47Z")
> [@pinocchio](#): > > I am still seeking clarification on the history part. Is it a deep copy of that or a pointer copy if we use deep copy? The history will not be copied, as you cannot call `copy.deepcopy` on a non-leaf tensor: ```python x = torch.randn(1, requires_grad=True) y = x + 1 copy.deepcopy(y) > RuntimeError: Only Tensors created explicitly by the user (graph leaves) support the deepcopy protocol at the moment ``` While `y` will be attached to the computation graph and will have a valid `.grad_fn`, you can only copy leaves as stated in the error message. If you want to keep the history, use `.clone()`, otherwise `.detach()` the tensor additionally to the `clone()` call. ---
**Author:** ![pinocchio](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/pinocchio/32/22093_2.png) [@pinocchio](https://discuss.pytorch.org/u/pinocchio)\ **Post date:** [June 18, 2020, 11:04pm UTC](https://discuss.pytorch.org/t/copy-deepcopy-vs-clone/55022/12 "2020-06-18T23:04:38Z")
> [@ptrblck](#): > > ```auto > > RuntimeError: Only Tensors created explicitly by the user (graph leaves) support the deepcopy protocol at the moment > > ``` I always wondered why that error appeared! Whats the choice for that semantics? ---
**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\ **Post date:** [June 19, 2020, 5:07am UTC](https://discuss.pytorch.org/t/copy-deepcopy-vs-clone/55022/13 "2020-06-19T05:07:48Z")
I don’t know, why the `deepcopy` isn’t supported (also wasn’t supported on `Variables`) and my best guess is that `clone()` or `detach().clone()` are a valid workaround and are also more explicit. ---
**Author:** ![pinocchio](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/pinocchio/32/22093_2.png) [@pinocchio](https://discuss.pytorch.org/u/pinocchio)\ **Post date:** [June 24, 2020, 5:30pm UTC](https://discuss.pytorch.org/t/copy-deepcopy-vs-clone/55022/14 "2020-06-24T17:30:06Z")
who would know why it’s not supported? ---
**Author:** ![jchiu](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/j/7c8e57/32.png) [@jchiu](https://discuss.pytorch.org/u/jchiu)\ **Post date:** [December 31, 2022, 4:50am UTC](https://discuss.pytorch.org/t/copy-deepcopy-vs-clone/55022/15 "2022-12-31T04:50:18Z")
Hi! This thread has been extremely useful but I ran into an issue with using `copy.deepcopy()` on a model inherited from `pl.LightningModule` where `"_trainer"` attribute was not `None`. It would throw the following error: `AttributeError: 'MyModel' object has no attribute '_parameters'` I noticed that setting it to `None`, running `copy.deepcopy()` and then resetting it would solve this issue. Is this an issue caused by PyTorch Lightning? ---
**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\ **Post date:** [December 31, 2022, 6:28am UTC](https://discuss.pytorch.org/t/copy-deepcopy-vs-clone/55022/16 "2022-12-31T06:28:29Z")
This issue might be specific to Lightning and seems to be related to [this one](https://github.com/Lightning-AI/lightning/issues/7505#issuecomment-1339718787). ---
**Author:** ![jchiu](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/j/7c8e57/32.png) [@jchiu](https://discuss.pytorch.org/u/jchiu)\ **Post date:** [December 31, 2022, 6:40am UTC](https://discuss.pytorch.org/t/copy-deepcopy-vs-clone/55022/17 "2022-12-31T06:40:33Z")
Got it. Will follow up on this discussion in the github issue thread! Thanks for your help. 🙂