How to reach model attributes wrapped by nn.DataParallel?

I have a custom nn.Module which uses another nn.Module called ‘rnn’. Before wrapping with nn.DataParallel I was able to reach it by model.rnn but after it raises AttributeError: 'DataParallel' object has no attribute 'rnn'

What is the way to reach model attributes inside DataParallel?

8 Likes

i come across the same problem as u met,how do u solve it,thank u

You can access them by using

model.module.rnn

but it’s not exposed in the public API, so it might change in the future.

18 Likes

When you wrap a model with DataParallel,
the model.module can be seen as the model before it’s wrapped.

You can use model.module.attribute to reach the model attribute like before.

11 Likes

What if I chage the attribute by this way? Will the models on other GPUs change synchronously?

3 Likes

Hello,
I saw a good project in github about dctts text to speach.
But this project is for only one gpu:

    device =  "cuda:0"

    graph = Text2Mel().to(device)
    # set the training flag
    graph.train()

i tried ‘dataparallel’ but i’m noob in pytorch and alway i have this error:

AttributeError: 'DataParallel' object has no attribute 'attention'

How i can use my 8 gpu? the Repo is:

I would appreciate any help, thanks!

1 Like