Data parallel of submodules?

I’m wondering if you use data parallel, will the sub module also be data parallel? For example, I write one model as A and in B I directly used A, when I use data parallel to wrap B, will A also be data parallel?

nn.DataParallel replicates the passed module on your GPUs, so if A is a member of B, both should run in parallel.