How to understand the layer object in the forward function

autograd
1

Hello Everybody,

I’m new to PyTorch and I have a question about how to understand the layer object in the forward function. Here is an example:

class Net(torch.nn.Module): 
    def __init__(self, n_feature, n_hidden, n_output):
        super(Net, self).__init__()  
        self.hidden = torch.nn.Linear(n_feature, n_hidden)  
        self.predict = torch.nn.Linear(n_hidden, n_output)  

    def forward(self, x):   
        x = F.relu(self.hidden(x))
        x = self.predict(x)           
        return x

net = Net(n_feature=1, n_hidden=10, n_output=1)

The self.hidden is defined as a Linear class object in the init(). Why this object (self.hidden) can accept x (self.hidden(x)) as an input? It’s not a function but an object! Is the forward() function in the Linear class called automatically here?
Thank you very much!

2

nn.Linear is also an nn.Module just like your Net.
Internally when you call Net(x) or self.hidden(x) the __call__ method is called, which registers hooks etc., and finally calls forward. You can check it out in the source file.

2 Likes
3

Thank you so much for your timely answer! It’s clear to me now.