I have built a DNN with only one hidden layer, the following are the parameters:
input_size = 100
hidden_size = 20
output_size = 2
self.linear1 = nn.Linear()
self.linear2 = nn.Linear()
x1 = F.leaky_relu()
#unimportant codes omitted
loss_function = nn.CrossEntropyLoss()
optimizer = optim.SGD(model.parameters(), lr=0.02)
normalized word vectors of size 100 from authoritative github are used as input
My purpose is to identify whether a word is an event. For example, ‘dought’ is an event but ‘dog’ is not.
After training, the 2-dimensional output tensors are almost the same (say,(-0.8,-1.20) and (-0.8,-1.21), (-0.2,-1.01) and (-0.2,-1.02)) even if the activation function and loss function are changed.
Could someone tell me the reason? I tried my best but failed to solve it.