I’m not entirely sure how
nn.Module works in detail, so I wanted to ask one question.
Are there any operations that are not allowed or should not be done in
forward in order to make
backward work properly?
For example (assuming my data is of shape
(n, 10), e.g.
n observations 10 features each) will backprop work just fine with such definition of forward function?
class SimpleNet(nn.Module): def __init__(self): super(SimpleNet, self).__init__() self.fc1 = nn.Linear(5, 2) self.fc2 = nn.Linear(5, 2) self.fc_final = nn.Linear(4, 1) def forward(self, x): x1_1 = F.relu(self.fc1(x.narrow(1, 0, 5))) x1_2 = F.relu(self.fc2(x.narrow(1, 5, 10))) return self.fc_final(torch.cat([x1_1, x1_2]).view(-1, 10))
Are there any particular limitations with regard to
forward? If there are some, I’d like to ask how to achieve an architecture similar to the one presented in a proper way.