How to use DataParallel with my model?

1

Here is the model that I want to use :

class Vgg16Slice(nn.Module):
    def __init__(self, requires_grad=False):
        super(Vgg16Slice, self).__init__()
        vgg_pretrained_features = models.vgg16(pretrained=True).features
        self.slice1 = torch.nn.Sequential().cuda(0)
        self.slice2 = torch.nn.Sequential().cuda(0)
        self.slice3 = torch.nn.Sequential().cuda(1)
        self.slice4 = torch.nn.Sequential().cuda(1)
        for x in range(4):
            self.slice1.add_module(str(x), vgg_pretrained_features[x])
        for x in range(4, 9):
            self.slice2.add_module(str(x), vgg_pretrained_features[x])
        for x in range(9, 16):
            self.slice3.add_module(str(x), vgg_pretrained_features[x])
        for x in range(16, 23):
            self.slice4.add_module(str(x), vgg_pretrained_features[x])
        if not requires_grad:
            for param in self.parameters():
                param.requires_grad = False

    def forward(self, X):
        h = self.slice1(X)
        h_relu1_2 = h
        h = self.slice2(h)
        h_relu2_2 = h
        h = self.slice3(h)
        h_relu3_3 = h
        h = self.slice4(h)
        h_relu4_3 = h
        vgg_outputs = namedtuple("VggOutputs", ['relu1_2', 'relu2_2', 'relu3_3', 'relu4_3'])
        out = vgg_outputs(h_relu1_2, h_relu2_2, h_relu3_3, h_relu4_3)
        return out

It has been taken from fast_neural_style from /pytorch/examples. When I use this model like this :

vgg = Vgg16Slice(requires_grad=False)
vgg = torch.nn.parallel.DataParallel(vgg).cuda()

On training, I get this error :

TypeError: __new__() missing 3 required positional arguments: 'relu2_2', 'relu3_3', and 'relu4_3'
Makefile:7: recipe for target 'main' failed
make: *** [main] Error 1

I tried editing the model by making top two slices as cuda(0) and bottom two as cuda(1), but that didn’t help. How can I make it work?

1 Like