Advantage of creating multiple layers with same arguments

Case 1:

class _ResGraphConv(nn.Module):
    def __init__(self, adj, input_dim, output_dim, hid_dim, p_dropout):
        super(_ResGraphConv, self).__init__()

        self.gconv1 = _GraphConv(adj, input_dim, output_dim, p_dropout)
        self.gconv2 = _GraphConv(adj, input_dim, output_dim, p_dropout)

    def forward(self, x):
        residual = x
        out = self.gconv1(x)
        out = self.gconv2(out)
        return residual + out

Case 2:

class _ResGraphConv(nn.Module):
    def __init__(self, adj, input_dim, output_dim, hid_dim, p_dropout):
        super(_ResGraphConv, self).__init__()

        self.gconv1 = _GraphConv(adj, input_dim, output_dim, p_dropout)

    def forward(self, x):
        residual = x
        out = self.gconv1(x)
        out = self.gconv1(out)
        return residual + out

As you can see in case 1, I have created 2 layers and calling them from forward. And in case 2, I’m creating only 1 layer as they take same arguments but calling that from forward twice to decrease the number of parameters. Is there any disadvantage of doing case 2?

@tanvir_14

In the second case, since your are reusing the same layer, your learnable parameters have been reduced.
Imagine this way, you are using the same filters on the image twice.

@anantguptadbl thanks for the answer…I got that part…but does it have any disadvantage (case 2) in the case of accuracy or something?

@tanvir_14 yes it will definitely have. The output of the two layers might be capturing some essence of the image by filters, but since you are using the same conv layer, it will be restrained to use the same filter to extract features out of the image for 2 layers

In ideal scenarios it is not a good idea to use Case 2

1 Like

@anantguptadbl thanks so much for the answer…I got ur point