I’m trying to solve a regression problem (~10 inputs, 1 output) and have based my code on curiosily’s tutorial (even though it’s for binary classification, it should work for regression after minor tweaking).
Now I’m looking to optimize the hyperparameters of my NN. I want some algorithm to help me optimize the:
- learning rate and momentum
- no. hidden layers, no. hidden units in each layer
- learning rate decay
- mini-batch size
I’m aware of optuna, ax, hyperopt, and ray. There could be others, but I’m keen to see if there are any packages particularly well suited to solving regression problems (and have good docs/tutorials/example code).