Choice of the hidden layers dimension

1

Hello ,
I’m training a forward neural network using [8,16,8] hidden layers , my input = 20 and my output = 14 and i’m getting a training accuracy = 51% and a test accuracy = 50% . My question how do I improve my accuracy and my choice of hidden layers.
Thank you.

2

Hi,

Whom, it’s very difficult to find the best choice of hidden layers. It’s like a lottery.
Though, I’ll write down what I know to improve feed-forward neural networks.

  1. As to input, is there any categorical variable? If so, give it a try to embed them using nn.Embedding layers. This technique is called Entity Embedding.
  2. Did you normalize continuous variables? Sometimes normalization has big effects.
  3. Did you try nn.Dropout or F.dropout to output of each layer except for the vary last layer? If you try this technique, remember to enlarge layer size. e.g. [8, 16, 8] → [50, 50, 50]
  4. Is your dataset large enough? Sometimes machine learning algorithms e.g. Random Forest and XGBoost/LightGBM are better than neural networks.