What is the difference __init__() and forward() in a network model?

PyTorch Live
1

I have the following code for a neural network. I am confused about what is the difference between the use of init() and forward() methods. Does the init() method behave as the constructor? If so, what is the significance of the forward() method? Is it necessary to use both while creating the network?

class MyNeuralNetwork(nn.Module):
    def __init__(self, input_size, hidden_size, output_size):
        super(MyNeuralNetwork, self).__init__()
        self.fc1 = nn.Linear(input_size, hidden_size)
        self.relu = nn.ReLU()
        self.fc2 = nn.Linear(hidden_size, output_size)
        self.sigmoid = nn.Sigmoid()

    def forward(self, x):
        out = self.fc1(x)
        out = self.relu(out)
        out = self.fc2(out)
        out = self.sigmoid(out)
        return out
2

__init__ is a constructor method used to initialize the parameters of the network. It is executed when an object of the class is created. For example, in PyTorch, this method is used to define the layers of the network, such as convolutional layers, linear layers, activation functions, etc.

forward is the method that defines the forward pass of the neural network. This method takes the input data and passes it through the layers of the network to produce the output. This method is executed whenever the model is called to make a prediction or to compute the loss during training.

In other words, __init__ sets up the network’s structure by defining the layers, while forward specifies how the data flows through the network. Both methods are required to create a neural network in PyTorch and serve different purposes.

1 Like