# How to reach model attributes wrapped by nn.DataParallel? **URL:** \ **Category:** Uncategorized\ **Created:** [March 26, 2017, 5:45pm UTC](https://discuss.pytorch.org/t/how-to-reach-model-attributes-wrapped-by-nn-dataparallel/1373 "2017-03-26T17:45:16Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![erogol](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/erogol/32/80_2.png) [@erogol](https://discuss.pytorch.org/u/erogol)\ **Post date:** [March 26, 2017, 5:45pm UTC](https://discuss.pytorch.org/t/how-to-reach-model-attributes-wrapped-by-nn-dataparallel/1373/1 "2017-03-26T17:45:17Z")
I have a custom nn.Module which uses another nn.Module called ‘rnn’. Before wrapping with nn.DataParallel I was able to reach it by `model.rnn` but after it raises `AttributeError: 'DataParallel' object has no attribute 'rnn'` What is the way to reach model attributes inside DataParallel? ---
**Author:** ![andyhx](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/andyhx/32/367_2.png) [@andyhx](https://discuss.pytorch.org/u/andyhx)\ **Post date:** [March 29, 2017, 1:18am UTC](https://discuss.pytorch.org/t/how-to-reach-model-attributes-wrapped-by-nn-dataparallel/1373/2 "2017-03-29T01:18:20Z")
i come across the same problem as u met,how do u solve it,thank u ---
**Author:** ![fmassa](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/fmassa/32/58_2.png) [@fmassa](https://discuss.pytorch.org/u/fmassa)\ **Post date:** [March 29, 2017, 1:22am UTC](https://discuss.pytorch.org/t/how-to-reach-model-attributes-wrapped-by-nn-dataparallel/1373/3 "2017-03-29T01:22:36Z")
You can access them by using ```auto model.module.rnn ``` but it’s not exposed in the public API, so it might change in the future. ---
**Author:** ![oneTaken](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/o/c89c15/32.png) [@oneTaken](https://discuss.pytorch.org/u/oneTaken)\ **Post date:** [October 24, 2017, 6:23am UTC](https://discuss.pytorch.org/t/how-to-reach-model-attributes-wrapped-by-nn-dataparallel/1373/4 "2017-10-24T06:23:27Z")
When you wrap a model with `DataParallel`, the `model.module` can be seen as the model before it’s wrapped. You can use `model.module.attribute` to reach the model attribute like before. ---
**Author:** ![MrTuo](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/mrtuo/32/4088_2.png) [@MrTuo](https://discuss.pytorch.org/u/MrTuo)\ **Post date:** [December 10, 2018, 1:12pm UTC](https://discuss.pytorch.org/t/how-to-reach-model-attributes-wrapped-by-nn-dataparallel/1373/5 "2018-12-10T13:12:16Z")
What if I chage the attribute by this way? Will the models on other GPUs change synchronously? ---
**Author:** ![Victor\_Maldonado](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/v/e19b73/32.png) [@Victor\_Maldonado](https://discuss.pytorch.org/u/Victor_Maldonado)\ **Post date:** [February 27, 2019, 2:06pm UTC](https://discuss.pytorch.org/t/how-to-reach-model-attributes-wrapped-by-nn-dataparallel/1373/6 "2019-02-27T14:06:39Z")
Hello, I saw a good project in github about dctts text to speach. But this project is for only one gpu: ```auto device = "cuda:0" graph = Text2Mel().to(device) # set the training flag graph.train() ``` i tried ‘dataparallel’ but i’m noob in pytorch and alway i have this error: ```auto AttributeError: 'DataParallel' object has no attribute 'attention' ``` How i can use my 8 gpu? the Repo is: > **[chaiyujin/dctts-pytorch](https://github.com/chaiyujin/dctts-pytorch)** > > The pytorch implementation of DC-TTS. Contribute to chaiyujin/dctts-pytorch development by creating an account on GitHub. I would appreciate any help, thanks! ---
**Author:** ![hobi](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/hobi/32/34260_2.png) [@hobi](https://discuss.pytorch.org/u/hobi)\ **Post date:** [March 2, 2021, 8:33pm UTC](https://discuss.pytorch.org/t/how-to-reach-model-attributes-wrapped-by-nn-dataparallel/1373/7 "2021-03-02T20:33:15Z")
I know this post is 3-4 years old, but if anyone runs into this problem hopefully this is of help. If you want to change an attribute in the case where you want to copy a layer from one model to the other, make sure you wrap the model your copying from in DataParallel (custom) also. I was also wondering if this was possible. Just make sure to also wrap the model that you are copying from in DataParallel. I made a mistake in not doing so, resulting in lot of wasted time… ```auto # simple fix for dataparallel that allows access to class attributes class MyDataParallel(torch.nn.DataParallel): def __getattr__ (self, name): try: return super(). __getattr__ (name) except AttributeError: return getattr(self.module, name) # def __setattr__ (self, name, value): # try: # return super(). __setattr__ (name, value) # except AttributeError: # return setattr(self.module, name, value) def load_weights(base_model_name, model, epoch): """ Loads previously trained weights into a model given an epoch and the model itself :param base_model_name: name of base model in training session :param model: the model to load weights into :param epoch: what epoch of training to load :return: the model with weights loaded in """ pretrained_dict = torch.load('{}/{}_{}_{}.pt'.format(args.model_path, base_model_name, epoch, args.lr))['state_dict'] model_dict = model.state_dict() pretrained_dict = {k: v for k, v in pretrained_dict.items() if k in model_dict} model_dict.update(pretrained_dict) model.load_state_dict(model_dict) return model model = r2plus1d50() model = MyDataParallel(model, device_ids=[0, 1]) # Copying the layer from epoch 13 to model epoch = 13 head_model = r2plus1d50() head_model = MyDataParallel(head_model, device_ids=[0, 1]) head_model = load_weights(base_model_name, head_model, epoch) # head1 is the name of the attribute that we want to copy over head = getattr(head_model, "head1") # set attribute to change the attribute setattr(model.module, "head1", head) ``` For some reason, I was not able to overwrite the setattr() method for a custom DataParallel class, which is why the model is accessed with model.module. Hope this helps.