When dealing with a small dataset and a complex model, what optimizer should i choose ?
SGD and lots of Dropout. SGD has demonstrated better generalizability, which is what you want. It just takes longer.
When dealing with a small dataset and a complex model, what optimizer should i choose ?
SGD and lots of Dropout. SGD has demonstrated better generalizability, which is what you want. It just takes longer.