Why NN doesnot learn when number of output nodes is larger than input nodes?

1

Hello! I am working on Neural Network for a regression problem. I have 4 inputs and 7 outputs. But it doesn’t learn at all anything. I am applying minmax normalization for input and output (0 to 1). Optimizer: Adam, learning rate = 0.0005. Do you have any suggestions, please? Thanks in advance!
This is the architecture I am using: class Net(nn.Module):
def init(self, n_in, n_out):
super(Net, self).init()
self.fc1 = nn.Linear(n_in, 32)
self.fc2 = nn.Linear(32, 64)
self.fc3 = nn.Linear(64, 128)
self.fc4 = nn.Linear(128, 64)
self.fc5 = nn.Linear(64, 32)
self.fc6 = nn.Linear(32, n_out)

def forward(self, x):
    x = F.relu(self.fc1(x))
    x = F.relu(self.fc2(x))
    x = F.relu(self.fc3(x))
    x = F.relu(self.fc4(x))
    x = F.relu(self.fc5(x))
    x = self.fc6(x)
    return x
2

Your model seems to be able to overfit random data as seen here:

model = Net(4, 7)
x = torch.randn(64, 4)
target = torch.randn(64, 7)
optimizer = torch.optim.Adam(model.parameters(), lr=1e-3)
criterion = nn.MSELoss()


for epoch in range(1000):
    optimizer.zero_grad()
    output = model(x)
    loss = criterion(output, target)
    loss.backward()
    optimizer.step()
    print("epoch {}, loss {}".format(epoch, loss.item()))

so you could check if changing some hyperparameters might help in your case.